• Fabian Wetzel
  • 2018-07-26
  • 11 min. read

Automating Jekyll-based web page build and deployments

What do I want to archieve?

This blog runs (currently) on Hexo but my German blog runs on Jekyll. In my last post I developed a way to use a Docker container for serving, building and deploying the Jekyll-blog on Windows. In case you do not know, Hexo and Jekyll take a bunch of blog posts written in Markdown together with some layout and compile them into a static web site. These files can then be hostet everywhere.

I like editing the markdown files and it is even possible in the browser on the github page. But actually compiling the blog is not possible directly in the browser and I want to change this. Currently, I always need to have my notebook or desktop PC ready for this. I use the Docker image jekyll/jekyll for the work but there is no need to run this container locally! Azure container instances can run this in the cloud for me as well and it is billed per second.

So there evolves my target model which I want to archieve:

  1. Push change of the blog content to Github repo (or edit directly)
  2. Github Webhook triggers an azure function
  3. The azure function starts a new container in azure
  4. the container…
    1. fetches the latest source from the git repo
    2. builds it with jekyll
    3. deploys it to the web server

The azure function is needed because the webhook cannot be customized enough to start a container instance directly. Also, I want to run a slightly different command based on the changed branch.

Solving it backwards

I think it is best to solve it backwards because I can always do one step before manually instead of automated. For example, I can trigger the container start via the command line before setting up the azure function. Also, the container setup without the azure function provides some value already. An azure function without the container does not.

Setting up a docker container for build and deploy

What I start with

…is this powershell command which uses either the azure cli or the new cloud shell if you like:

1
az container create --resource-group cicd --name blog-en-cicd --image jekyll/builder --restart-policy never--gitrepo-url https://github.com/fabsenet/blog_de.git --gitrepo-mount-path /srv/jekyll --gitrepo-revision master --command-line "jekyll build"

See also the docs for az container create!

This command does most of what I want already. It will create a container with the name blog-en-cicd in an azure ressource group named cicd (this must exist already). It will also use the gitrepovolume function to clone my blog source with the specified revision (=branch, hash, tag, …) in the specified directory /srv/jekyll.

The restart policy never will start this container at most once. Either it works or not. The most likely case of failing is me messing something up and this will not change on a retry. Also the idea is to start the container, build and deploy exactly once and get rid of it. Per minute billing is perfect for this as well. I estimated that the compilation even for heavy blogging will hardly cost me more than a single cent.

What is missing from here is a way to publish the resulting html(+other files) and most probably a way to store the secret somewhere in there because the blog source is public but my hosting server is not!

Viewing outcomes of container runs

To view the details of the last run, I use az container show --resource-group cicd --name blog-en-cicd -o table to take a look at the ProvisioningState and I use az container logs --resource-group cicd --name blog-en-cicd to view the output of the commands that were run in the container. In the example this was only git status to make sure, the revision command works as expected:

powershell result for az container show and az container logs

Setting up the deployment

Based on the results of the last blog post, I like to simply repeat the command concatenation of build and uploading.

1
az container create --resource-group cicd --name blog-en-cicd --image jekyll/builder --restart-policy never --gitrepo-url https://github.com/fabsenet/blog_de.git --gitrepo-mount-path /srv/jekyll --gitrepo-revision master --command-line "bash -c 'jekyll build && echo upload command here'"

Getting the quoting of quotes in this command right gets harder and harder but the command looses its value fast anyway because to automate it, it actually has to be converted in the deployment of a template to azure anyway.

converting the az command to a deployment template

I am sure, there is also an az-command for this, but this time, I opened the azure portal and navigated to my container instance, selected automation scripts and found the following on the template tab:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "containerGroups_blog_en_cicd_name": {
            "defaultValue": "blog-en-cicd",
            "type": "String"
        }
    },
    "variables": {},
    "resources": [
        {
            "comments": "...",
            "type": "Microsoft.ContainerInstance/containerGroups",
            "name": "[parameters('containerGroups_blog_en_cicd_name')]",
            "apiVersion": "2018-04-01",
            "location": "westeurope",
            "tags": {},
            "scale": null,
            "properties": {
                "containers": [
                    {
                        "name": "[parameters('containerGroups_blog_en_cicd_name')]",
                        "properties": {
                            "image": "jekyll/builder",
                            "command": [
                                "bash",
                                "-c",
                                "jekyll build && echo upload command here"
                            ],
                            "ports": [],
                            "environmentVariables": [],
                            "resources": {
                                "requests": {
                                    "memoryInGB": 1.5,
                                    "cpu": 1
                                }
                            },
                            "volumeMounts": [
                                {
                                    "name": "gitrepo",
                                    "mountPath": "/srv/jekyll"
                                }
                            ]
                        }
                    }
                ],
                "restartPolicy": "Never",
                "osType": "Linux",
                "volumes": [
                    {
                        "name": "gitrepo",
                        "gitRepo": {
                            "repository": "https://github.com/fabsenet/blog_de.git",
                            "directory": ".",
                            "revision": "master"
                        }
                    }
                ]
            },
            "dependsOn": []
        }
    ]
}

If you want to use this template yourself, you obviously have to replace all of my values with yours or you will actually build my blog.

Building the azure function

functions basics

So I have to write the azure function in a way to deploy this template next. I have never used azure functions before, so there might be an easier or different way to the stuff. You have been warned! I loosely follow the quickstart sample from the docs.

I need a new storage account for the azure function where they can keep their state and stuff:

1
az storage account create --name cicdfunctionsstore --resource-group cicd --sku Standard_LRS --kind StorageV2

Next I create the azure function app based on the quickstart sample:

1
az functionapp create --deployment-source-url https://github.com/Azure-Samples/functions-quickstart --resource-group cicd --consumption-plan-location westeurope --name cicdfunctions --storage-account  cicdfunctionsstore

This works so far, now I need to actually edit the functions. I follow the installation guide for the azure functions extension in visual studio code for this. After login, I can already browse the list of my functions:

azure functions list in VS code

From there, I selected create new function... and I was guided through some choices to init my workspace, selecting the programming language (c#) , the trigger type (http), the authentication and finally a name. I then clicked deploy to function app.... This actually overwrote the quickstart functions, so creating them in the first place was not really neccessary, I guess. This is all really impressive but I would like to be able to run the function locally as well. I switched to my functions directory locally and executed func start --build. Thats it, I then opened http://localhost:7071/api/TriggerJekyllBuild?name=Fabse and it worked.

Firefox showing the local azure functions call

making the actual function

Now I need to put the container template deployment in the function somehow. I provide only a small summary of what I did here because the post is sooo long already:

  • added the template as an embedded ressource
  • added the boilerplate class from the portal (automation script) and added all missing values:
    • added the nuget packages the boilerplate class requires (dotnet add package ...)
    • registered an app identity which is allowed to deploy stuff. (docs)
    • copied various IDs and secret in the code from that app identity, azure AD, and so on.

…and it worked after only 20 minutes of tinkering. Nice!

Hooking the azure function to GitHub webhooks

This is probably the easiest part. I go to my repo on GitHub. You should probably go to yours. Then settings, then webhooks, the Add webhook. Then I added my azure function url and selected JSON as payload:

GitHub Add Webhook dialog

I also secured everything with some signature verification from a very good blog post, except it did not work because it reads the body as a string and converts it back to a byte-array and it maybe does not use the same encoding for both steps? I converted it to reading the body directly as a byte[] and converted that later to a string.

Key points:

1
2
3
4
5
6
var ms = new MemoryStream();
request.Body.CopyTo(ms);
var requestBytes = ms.ToArray();

ms.Position = 0;
var requestBody = new StreamReader(ms).ReadToEnd();

What kept me from finishing this earlier was that the container instance sometimes is stuck in the state creating or pending and nothing happens. I tracked that down to the git revision used for building the blog. Simply providing master works, but GitHub push events state them as refs/heads/master. The long version stops the container instance from deploying. I think it has to do with the slashes. This code fixed it for me:

1
2
3
4
if(branch.StartsWith("refs/heads/"))
{
    branch = branch.Substring("refs/heads/".Length);
}

Final thoughts

It was actually a fun ride to bring so many bleeding edge technologies and services together! So much was new to me and I managed to get it working anyway. I am feeling very very successful right now!

#automation #azure #blog #docker #My Setup