Introducing a whole new Blacksmith

In honor of NKO we went back to the roots of the Node.js community, experimentation, to completely rewrite blacksmith from the ground up. Over time blacksmith grew as we used it for docs.nodejitsu.com and the Nodejitsu blog, but there were some issues that never quite got resolved:

  1. Use plates over weld.
  2. Reusing HTML (such as layouts).
  3. Better documentation of rendering conventions and settings.

So read on to learn about how blacksmith is back and better than ever. In fact, it is used to render this very blog!.

When rewriting blacksmith there were some very clear goals. This post will explore each of these goals and how they were achieved.

  1. Reuse as much HTML markup as possible.
  2. Make rendering as generic for flexibility of sites and layout.
  3. Make writing content as simple as possible.
  4. Ensure metadata is as flexible as layout components.
  5. Have data structures reflect the rendering hierarchy.
  6. Do not care about backwards compatibility.

Reuse as much HTML markup as possible.

In previous versions of blacksmith each page that was rendered needed a fully-formed HTML file that included the template of what was to be rendered. This caused a lot of duplication of HTML in blacksmith themes.

In the new version of blacksmith your site is broken out into a set of directories for each of the components used when rendering your site:

/site-name
  #
  # Settings for this blacksmith site.
  #
  .blacksmith
  /content
    #
    # Actual Markdown content to render.
    #
    /posts
      a-post.md
      another-post.md
  /layouts
    #
    # Layouts to use for pages. You can specify 
    # multiple layouts, but default.html is ... the default.
    #
    default.html
  /metadata
    #
    # Metadata entities which can be reference in content metdata
    #
    /authors
      author-name.json
  /partials
    #
    # HTML for partials inside of pages
    #
    post.html
    sidebar.html
  /pages
    #
    # Metadata for rendering specific pages
    #
    index.json
    post.json
  /public
    #
    # Any additional files for viewing the site. All
    # rendered HTML will be placed here. e.g.:
    #
    /css
      styles.css
    /img
      favicon.png

Breaking changes: /theme will no longer be respected when creating your site. You will need to break everything into layouts and partials.

Make rendering as generic for flexibility of sites and layout.

In previous versions of blacksmith the page article was special. This greatly limited how generic your site could be and what content you could generate a site from.

In the new version of blacksmith each rendering component has it's own metadata that allows you to customize what content is rendered and how it is rendered:

  {
    //
    // Specifies the layout for the page. 
    // Default: Layout for the site.
    //
    "layout": "custom-layout-for-page",

    //
    // Specifies a mapping of HTML ids to partial(s). If "content" is specified
    // then it is appended after the rendered markdown content.
    //
    "partials": {
      "html-id": "partial-name",
      "another-id": ["multiple-partials"]
    },

    //
    // Case 1: Rendering content with a partial
    //
    "content": "custom-partial"

    //
    // Case 2: Consolidating multiple content sources in a list.
    //
    "content": {
      "list": "post",
      "truncate": true,
      "limit": 20
    }
  }

Breaking changes: You will need to specify JSON files in /pages for each type of content you wish to render.

Make writing content as simple as possible.

In previous versions of blacksmith each page required a content.md and page.json file. This extra step was unnecessary and cumbersome for specifying page metadata.

In the new version of blacksmith page metadata is stored in link definitions in your Markdown files prefixed with meta:. Because of a small limitation in the Markdown format you must use the following syntax to specify metadata:

  [meta:author] <> (Author Name)
  [meta:title] <> (A Really Long Title Different from the Filename)
  [meta:nested:values] <> (Are also supported)

Also in the previous version of blacksmith content references (i.e. files you wish to insert into your content) were accomplished by rendering the DOM and replacing <a class="snippet">... with the file contents.

Since rendering a full DOM is an expensive operation the new version of blacksmith does this with simple string replacement:

  /content
    /dir-post
      index.md
      whatever.js
  This is a piece of markdown content to be rendered later. It has a reference to the file "whatever.js"

  <whatever.js>

Breaking changes: You will need to move all values in page.json files into Markdown link definitions and update all content snippet usage.

Ensure metadata is as flexible as layout components.

In previous versions of blacksmith the only metadata that could be rendered into pages was stored in /authors. This greatly limited the flexibility of rendering partials in the new layout system.

In the new version of blacksmith metadata is divided into entities which may be referenced in page metadata via filename. For example: if you wish to store metadata about an author it is stored in /metadata/authors/name.json:

/metadata/authors/charlie-robbins.json

  {
    "name": "Charlie Robbins",
    "email": "charlie@nodejitsu.com",
    "github": "indexzero",
    "twitter": "indexzero",
    "github-url": "https://github.com/indexzero",
    "twitter-url": "http://twitter.com/indexzero",
    "location": "New York, NY"
  }

Will be loaded into a page by simply referencing it in any Markdown content file:

  [meta:author]: <> (Charlie Robbins)

Breaking changes: You will need to move all author metadata from /authors to /metadata/authors.

Have data structures reflect the rendering hierarchy.

When designing a layout system it is important to pay close attention to the internals to make sure future feature additions are simple to implement. Lets examine how the internals of blacksmith work:

  • Rendering starts in /my-site:

To start the rendering process simply install blacksmith and invoke it from the CLI:

  $ npm install blacksmith -g
  $ cd /path/to/my-site
  $ blacksmith
  • Load all HTML and rendering settings: blacksmith will load everything under /my-site:
  /my-site
    .blacksmith
    /layouts/*
    /metadata/*
    /partials/*
    /page/*
  • Read everything under /content:

Before layout, page, and partial rendering can take place all content (i.e. Markdown and supporting files) must be known to blacksmith. In our example how can we render the page represented by /pages/index.json without all rendered content? Site.prototype.readContent recursively reads /content building this data structure:

/content
  /posts
    a-post.md
    another-post.md
    /dir-post
      file1.js
      file2.js
      index.markdown
{
  posts: {
    "a-post": { _content: { html: "..", markdown: "..", metadata: { ... } } },
    "another-post": { _content: { html: "..", markdown: "..", metadata: { ... } } }
    "dir-post": {
      _content: {
        html: "HTML rendered from Markdown", 
        markdown: "Markdown in index.markdown",
        metadata: {
          "page-details": {
            source: "/dir-post/index.markdown",
            title: "Title of page in metadata" || "href of page",
            date: "Saturday, November 10, 2012",
            href: "/dir-post",
            "files": {
              "js": [{ "filename": "file1.js", "url": "/full/path/to/file1.js" }],
              "img": [{ "filename": "img1.png", "url": "/full/path/to/img1.png" }]
            }
          }
        }
      },
      _files: ['file1.js', 'file2.js']
    }
  }
}

As you can see the data structure resulting from the rendered content reflects the file structure. In this way any additions to blacksmith will have a simple data structure to extend and work with.

  • Render all pages into /public:

Once all of the content has been loaded we iterate over the content and render a page for each _content item in the JSON structure writing the associated file to /public/[path].html.

  • Serve your rendered static site:

And that's it. In the new blacksmith you can see that the rendering is more simple and hierarchical than in the previous version allowing you to build any kind of site you want! You can serve your blacksmith site with any static file serve. Why not tryout http-server?

  $ npm install http-server
  $ cd /path/to/my-site
  $ http-server -e html

Do not care about backwards compatibility

This bears repeating: blacksmith@1.0.0 has zero backwards compatibility with previous versions of blacksmith. So make note of all of these changes and read all the documentation. Rest assured though, wow that blacksmith has reached 1.0.0 efforts will be made to ensure all API changes as minimal.

You can read all of details about how to use blacksmith@1.0.0 on the Github page.