Enabling HSTS for Fabse.net

Today, I read about HTTP strict transport security on troyhunt.com and I decided to enable it for fabse.net as well. After running this complete website over HTTPS for some time now, I decided to go the next step and to (technically) tell all browsers that this website will only ever work over a secure connection. Hanselman has a great blog post on how to do this correctly on IIS.

At last, I also added my site to to the official preload list to get it directly included in future browser releases.

Installing Windows 10 IoT Core using a Hyper-V Windows 10 machine

So I finally came around to play with my raspberry pi 2 a little bit and my first objective was to install Windows 10 IoT core on it. The full getting started tutorial has all the needed steps written.

After registering and accepting the EULAs (important! you will simply not see the downloads otherwise!) I noticed the first obstacle for me. They require you to copy the data to sd card using a Windows 10 hardware machine. I have only one Windows 10 machine which is running in Hyper-V.

So I accepted the challenge :) This article suggested, it may at all be possible to copy the image to the sd card using Windows 7 or Windows 8.

I made a new folder on my Desktop PC (D:\win10-on-rpi) and extracted the flash image there (Flash.ffu). Then I copied some data from the Windows machine into this folder:

  • Dism.exe (from C:\Windows\System32)
  • DismApi.dll (from C:\Windows\System32)
  • All content inside C:\Windows\System32\Dism

Important is to copy the content from the Dism folder directly into folder and not into a subfolder. This is not intuitive, but works.

windows10-iot-raspberry-pi2-flash-folder

At this point I opened a powershell command prompt as an administrator and executed the Dism-command.

D:\win10-on-rpi> ./dism.exe /Apply-Image /ImageFile:Flash.ffu /ApplyDrive:\\.\PhysicalDrive7 /SkipPlatformCheck

windows10-iot-raspberry-pi2-flash-folder-dism-completed

Building a RAID 0 with 2 SSD Drives (and showing off)

My desktop PC at home is utilizing a 256GB SSD as its system drive for quite some time now and I enjoyed it. I measured the speed of it using HD Tune:SAMSUNG_SSD_830_SeriesIn the past, I had used another SSD of the same size for a fast external USB 3 drive to host a virtual machine but I did not needed the VM anymore and so the drive was unused now. I installed it into my PC as an internal drive and measured its speed (of course!):

Samsung_SSD_840_EVO_250GNext thing was backing up the complete system drive to network storage.Before I actually deleted the data from the first drive, I tried a full restore to the second drive, which took some time but worked well.

Then I felt comfortable enough to go ahead and to setup a RAID 0 with both drives. This was the time, were I noticed my first SSD was not connected to the Intel RAID chipset but to the secondary SATA chip on my board. A Marvel-something. I switched some drives around until I had both SSDs connected to the RAID chipset. Setting up a RAID 0 is straightforward. Next thing was doing the restore and resize of the system partition to the new RAID volume. And I measured it again:

Raid_0 (Samsung 930+940)This measured speed looks very impressive and it is significantly higher than the speed of both single drives combined. It may has to do with me using the Marvel chipset in the first measurement. Maybe the INTEL chipset is doing some performance magic on the cover as well? I do not know and I frankly do not care.

For what it’s worth, the actual perceived performance gain compared to a single SSD drive is next to nothing but it is definitely not slower than before and the system partition has now much more free space than before. The additional free space without partition/drive borders in between it is the real gain here for me.

Pitfalls of String to Number/Date Conversions

Having a string and converting it to a Number is easy, isn’t it?

Convert.ToDouble(string)

But what does it actually print on the console? As usually, that depends! It depends on your own user profile and its culture. Are you using a culture, where “.” (dot) is a decimal divider or a thousands separator?

So the example above will print “1.23” if you have an English culture setting or it will print “123” if you have a German culture. So that is most probably not what you want. What you want is consistency and predictability.

As a kind of an anti pattern (which I saw already way to often) you could replace dot with comma or comma with dot depending on what you want and you may get it. I am not showing samples on purpose here, but the key take-away is to not use string replacings for this purpose!

So what should you do instead? .Net has a good implementation of Cultures so it know very well how to handle all of this, the only thing you have to actively do is telling it, what culture it should use. If you do not provide a culture, it implicitly uses the CurrentCulture. So, the first example written in a more explicitly way would be:Explicit culture usage

So all you are left to decide is which Culture should be used, because the default is derived from the actual user context and will therefor be different for every user/region. In the case of the Console.WriteLine this might be fine, because you may want to show the number to the user in the way he is used to read numbers but the Convert.ToDouble use-case is different. You have to decide the culture to use based on the origin of the value and in my example it is a constant value in the program but it could come from a file or a web service or whatever or from user input.

To make my example work the same for every user, I am using the invariant culture:Convert.ToDouble with invariant culture

The invariant culture is sort of a baseline culture and it is mostly equal to the English culture. As a last bonus, here are some ways to create cultures:Some ways to get CultureInfo - Instances

Using [Obsolete] to handle legacy code

I frequently use the ObsoleteAttribute in legacy, spread-out codebases and I want to tell you how and why. But at first, what is the ObsoleteAttribute (MSDN)?

TL;DR; I like to use the obsolete attribute to transition implementations and signatures in a smooth way.

The attribute has 3 different constructors. The first one has no parameters at all. My advice here is to never use it ever because it does not tell why you used it.

The second constructor takes one string parameter. This is a message meant for others developers. If you use or try to use a method or class tagged with this attribute you will see the message and it will generate a compiler warning. I always use this message to explain “why?” and “what’s next?”. This means “Why is this message obsolete?”. It often is because the method has a vague naming or an obscure handling of some edge cases. But I see it as a good practice to always tell the caller what he should do next. So I always end the message with “please use the other method overload with parameter xyz.” or “please use method xyz instead”.

ObsoleteAttribute_in_VisualStudio

So why do I not simply refactor the method name and/or signature? Most legacy projects tend to have everything as public, have no unit tests and reuse code in several solutions in a mix+match style. It is hard to be absolutely certain to really find all callers. Some projects tend to do partial deployments as well. This means to replace only a subset of the assemblies in a deployment. This is most often done to save time. So as a baseline, I only refactor it, if I am sure it is safe and most of the time I cannot guarantee that. My next best way is to mark the method as obsolete and create a fixed version of it and then redirecting the caller using the message in the obsolete attribute.

But who cares about compiler warnings?

This is, where the last constructor comes into play. You can provide a “true” after the message and the warning becomes an error message which breaks the build. This is the route I take if I am fairly sure, I changed all callers. There is one benefit over simply removing the old method. The method with its implementation gets still compiled.

This helps in the case of an old assembly of a caller which was compiled against the old method which now loads the new library in the mixed deployment scenario. In the case of the obsolete attribute the actual method is still fully compiled and usable so the old process still works. If I had deleted the method instead, I would now receive a MissingMethodException.

And after some weeks/month I simply remove old obsolete methods and I can be sure, there are no callers left.

So in summary I like to use the obsolete attribute to transition implementation and method signatures in a smooth way.

Linq: Finding the inverse operation to SelectMany()

I am searching for the inverse operation to the SelectMany() linq method. But at first, we have to understand the SelectMany() method. Consider a simple array containing arrays of numbers and we do SelectMany() on them:

int[][] arrayOfArrayOfInts =
{
    new[]{1,2,3},
    new[]{20,21,22},
    new[]{99}
};

var numbers = arrayOfArrayOfInts.SelectMany(ints => ints);
foreach (var number in numbers)
{
    Console.Write("{0}, ", number);
}

This will output the single list of numbers:

1, 2, 3, 20, 21, 22, 99,

For the inverse operation, we have this single list of numbers and are trying to put them in different lists to build the given groups above again. Obviously we need some kind of way to decide, whether a given number should be added to the current list or should produce a new list. To solve this, we could do the integer division by 10 in a group by on all numbers, like so:

int[] numbers = {1, 2, 3, 20, 21, 22, 99};

var groupedNumbers = numbers
    .GroupBy(num => num/10)
    .Select(group => group);

foreach (var groupedNumber in groupedNumbers)
{
    Console.Write("Group: ");
    foreach (var num in groupedNumber)
    {
        Console.Write("{0}, ", num);
    }
    Console.WriteLine();
}

This outputs exactly what I wanted:

Group: 1, 2, 3,
Group: 20, 21, 22,
Group: 99,

So this article is done? Not quite yet, because I maybe oversimplified my actual task at hand :-)

The way my actual task works is that I only know whether a given number belongs to the list or should start a new list by looking at the previous number. We could express this as the following predicate:

private bool BelongsToSameGroup(int previousNumber, int currentNumber)
{
    return previousNumber + 1 == currentNumber;
}

The signature of the method is important as it needs not only the current item but also the previous item of the list! This is the point where GroupBy() is of no help anymore and where I got stuck for some time.

The Linq method I am searching should have a signature to something like this:

IEnumerable<IEnumerable<T>> 
    InverseSelectMany<T>(IEnumerable<T> source, Func<T, T, bool> belongsToSameGroupPredicate)
    {...}

Knowing the correct target is 90% of the solution. So having found the signature, I came up with the following solution:

static class InverseSelectManyLinqExtension
{
    public static IEnumerable<IEnumerable<T>> InverseSelectMany<T>(this IEnumerable<T> source, Func<T, T, bool> belongsToSameGroupPredicate)
    {
        var internalList = new List<T>();
        T lastItem = default(T);
        bool isFirstRun = true;
        foreach (var item in source)
        {
            if (!isFirstRun && !belongsToSameGroupPredicate(lastItem, item))
            {
                //start new group and return current list
                yield return internalList;
                internalList = new List<T>();
            }

            //add item to current list
            internalList.Add(item);
            lastItem = item;
            isFirstRun = false;
        }

        //return last list
        yield return internalList;
    }
}

This will build up an internal list of items as long as they belong together and will yield the list, if they do not belong together anymore.

The usage looks like:

int[] numbers = {1, 2, 3, 20, 21, 22, 99};

var groupedNumbers = numbers.InverseSelectMany(BelongsToSameGroup);

foreach (var groupedNumber in groupedNumbers)
{
    Console.Write("Group: ");
    foreach (var num in groupedNumber)
    {
        Console.Write("{0}, ", num);
    }
    Console.WriteLine();
}

This does now exactly what I needed :-)

Encrypt All The Things … and this blog

Google started to favor secure sites in search results. “Encrypt all the things” is trying to create a certificate authority to provide free SSL certificates. And last but not least are all the news stories about government agencies doing evil stuff here and there. This was enough motivation for me to secure my blog with https as well.

My blog was running on a subdomain (blog.fabse.net) and there was a redirect from the main domain. To secure this, I would have needed a certificate which would cover

  • fabse.net
  • www.fabse.net
  • blog.fabse.net

Because of the non-standard subdomain “blog” you need either a wildcard certificate or a multi-domain certificate and as it turns out, both are quite expensive. The reasonable priced certificates will only secure a single domain with and without the www, so I had to move my blog.

This was a thing, I had planned anyway for a while. Because I want my blog to be only one part of my web site and this should be reflected in the URL as well. So long story short, the plan of action was:

  • move the blog from blog.fabse.net to fabse.net/blog
  • make sure, all old URLs still work
  • get a https certificate and make it the default

Moving the blog was easy. I reconfigured the IIS web site binding, moved the wordpress code in a sub folder and changed the wordpress settings. It really took not more than 5 minutes.

Making sure all old URLs still work was the part I was most afraid of. I used URL rewriting in the IIS to issue permanent redirects for everything from blog.fabse.net/* to fabse.net/blog/*. This *HAS* to work, because otherwise I would loose all subscribed readers of the feed. And it does work, so everything is fine Smiley

Getting the certificate was easy. For now, it is a 30-day free certificate from psw.net. I first wanted to make sure, everything works as expected before I order a real certificate.

The real pain point is that a secure site is only allowed to load secure resources. I had to find and fix all occurrences of this issue.

To finalize the secure setup, I configured IIS to be even more secure including the use of forward secrecy and an URL redirect from http to https. The result is a very secure blog reading experience Smiley

image

ncgsim – A Network Creation Games simulator

You did not read much from me here for the last months, did you? I was coding a simulator for Network Creation Games and I was writing my master thesis about it as well. Now it’s done. Mostly.

What is Network Creation Games you ask? It is a game inspired by the internet itself. There is a graph and each node is a player. Every player has to build edges or use edges from other players to connect to all other player. Operating an edge introduces costs and using the edges from others has costs as well. Each player has the target to optimize their own costs to a minimum. The game ends, when no player is doing anything anymore.

Building a simulator for this was an interesting challenge because I decided to not use my well-known technology stack of .Net/C#/ASP.Net but to do a single page app built with AngularJS, Bootstrap and TypeScript. It was interesting for me and it had quite a learning curve as well. What I missed the most was Linq.

I was trying out Github for the hosting of the repository and issue tracking and I was doing real test-driven development and feature branches. It was lovely Smiley At one point, I had the idea to swap out one graph implementation with another. I did that and after I was done, all lights where green again and I felt a strong confidence, that I did not broke anything.

You can check it out yourself. It runs as an azure website on http://ncgsim.azurewebsites.com and the repo is at https://github.com/fabsenet/ncgsim

Let me know, what you think!

Changing Folders In PowerShell

I really like PowerShell more and more lately. So at no surprise to me, I get better at using it every day. Today I made my own level-up at folder switching. Consider the following folder structure as an example:

sample_structure

So there are two alpha folders inside my samplefolder containing another folder each. So my PowerShell command line is at the samplefolder and I want to get into the gamma folder. I would have done the following in the past:

D:\samplefolder> cd<tab><tab>
D:\samplefolder> cd .\alpha2<enter>
D:\samplefolder> cd <tab>
D:\samplefolder\alpha2> cd .\gamma<enter>
D:\samplefolder\alpha2\gamma> <done!>

As you may have noticed, I represented the special keystrokes in angle brackets. After doing this first way of navigating for many years now (because it works in cmd.exe as well) I learned, you could provide a filter expression before pressing tab:

cd a<tab><tab>

This would only go through all possible subfolders starting with an „a“. Of course, in this sample, this provides no use, but you can use the star as well:

cd *2<tab>

This expands to:

cd .\alpha2

Nice! We saved a tab keystroke, but we can do even better. If you are certain, your filter will correctly expand, you do not have to press tab first, but you can simply press enter:

D:\samplefolder> cd *2<enter>
D:\samplefolder\alpha2>

Wow! But we can do even better but before, we have to go one step back: What you also could do is changing into more than one directory deep at a time:

D:\samplefolder> cd .\alpha2\gamma<enter>
D:\samplefolder\alpha2\gamma>

And as you might already guessed by now, the filters work here as well:

D:\samplefolder> cd a*2\g*<enter>
D:\samplefolder\alpha2\gamma>

But I left the most crazy example for last: Consider the case, where you want to change to the gamma folder, but you do not know the correct alpha folder. You don’t have to!

D:\samplefolder> cd *\gamma<enter>
D:\samplefolder\alpha2\gamma>

PowerShell is smart enough to change into the right directory for you. Impressive!

Most of this stuff shown does not work in cmd.exe but why would you want to use cmd.exe anyway anymore?

On The Technical Health Of Fabse.Net

As some of you noticed, my blog was not available for about a day. There were actually two unrelated issues in a row affecting the availability of my blog.

The first one was related to Windows Update on my VM. There was some updating done but there was an outstanding restart. This somehow broke the connectivity between WordPress and MySQL. I am using WP Super Cache as a plugin for WordPress to cache all pages locally and so this database issue was going unnoticedd to most users except the Googlebot. It looks like requesting the robots.txt is not cached at all by WordPress. Google Webmaster Tools was sending emails to me notifying me about the issue of not being able to access the robots.txt file and postponing the indexing. At this point I simply restarted the virtual machine and it worked again. Have you tried turning it off and back on again?

My blog runs on a small VM in Microsoft Azure and the Azure storage in West Europe had a reduced availability yesterday affecting the VMs as well. Whatever reduced availability means. As a matter of fact, my VM was listed as running, but I could neither access my site nor could I RDP into it.

Microsoft requires at least two running VM instances in an availability set for their SLA to kick in. I am running only a single one to safe money. So I think it was actually my own fault/risk.

Regarding the near feature, I want to look into increasing the availability while reducing costs at the same time. I will definitely write about it here.