This blog shares one method of inserting images with captions without using any plugins or tags in Markdown.

This idea is borrowed from [1]. Insert images with captions like this:

!["dominating_sets_example2"](http://sparkandshine.net/wordpress/wp-content/uploads/2016/02/dominating_sets_example2.png)
*Fig. 2: The minimum dominating set of a graph*

# OR

<img src="http://sparkandshine.net/wordpress/wp-content/uploads/2016/02/dominating_sets_example2.png" alt="dominating_sets_example2"/>
*Fig. 2: The minimum dominating set of a graph*

The above text is rendered as HTML:

<p>
    <img src="http://sparkandshine.net/wordpress/wp-content/uploads/2016/02/dominating_sets_example2.png" alt="dominating_sets_example2"/>
    <br>
    <em>Fig. 2: The minimum dominating set of a graph</em>
</p>

Use adjacent selectors to customize the style and add it to your style.css, for instance:

/* for image caption */
img + br + em {
    font-style: normal;
    display: inherit;
    text-align: center;
    font-size: 90%;
}

The selector img + br + em matchces if img, br and em share the same parent (here is p) in the document tree and img directly precedes br, following up em. Refer to adjacent selectors on w3.org for the detailed description.

After publishing, it looks like:

Fig. 2: The minimum dominating set of a graph

References:
[1]StackOverflow: Using an image caption in Markdown Jekyll