Markdown can affect performance

Markdown is a very popular input format for static site generators, that's why Nikola has supported it since very early in life, even if I prefer reSt most of the time.

One thing that has surprised me a while ago is that, considering how minimalistic markdown is, and how little it does, it can be pretty slow to process. But it turns out this is not completely markdown's fault, but that the python markdown implementation is really, really, really slow.

How slow? Let's check it out. All benchmarks in this post were done in my notebook, a nice Asus Zenbook UX305, with a fast SSD and plenty of RAM. In all cases the builds were done using no parallelization.

The test site is an empty site to which I added 1000 copies of a simple, not too large markdown file (our theming guide, converted from reSt using pandoc).

As a baseline, a site with 1000 copies of the original reSt theming.txt builds in 126 seconds.

The markdown version of that site? It gets progressively slower as files build, so I suspect it leaks something between builds, and finishes in ... 1584 seconds. That is over 12 times slower than the reSt compiler.

But you don't want to switch to reSt? No problem! We support more markdown compilers than we probably should, so you can just choose which one you prefer.

Here is a chart showing the performance of each (HTML and reSt added as reference):

Seconds to build (bigger is worse) 00100100200200300300400400500500600600700700800800900900100010001100110012001200130013001400140015001500158472.67350427350428267.00000000000006194127.80512820512821492.28773310023314126182.93675213675215503.3090034965035110238.06837606837607505.9022435897436564293.20000000000005513.357808857808934348.331623931624518.220134032634132403.46324786324783518.544289044289125458.5948717948718519.678831585081614513.7264957264957521.4616841491842Seconds to build (bigger is worse)markdownkramdownrestpandoccommonmarkmistunemistune+cythonmisakahtml

There are just a few disadvantages to using any of the alternative Markdown compilers:

  • They are less tested, because fewer people use them (and if you use them, that changes!)

  • They may lack a specific feature of the "standard" markdown compiler, such as extension support (in which case, file a bug with us!)

  • The produced output may differ from the "standard", but that happens :-)


Comments powered by Disqus