Writings

For most of 2020 I was planning to create a dedicated website for my “language engineering with MPS” tutorials that I streamed on Twitch at the end of 2018 and early 2019. As part of creating a website for them I also wanted to update the tutorials to more recent MPS version as it was still stuck on 2019.1. Through 2020 I didn’t find the time to do it for various reasons but in November I finally got around doing it.

Last year I started a stream series about JetBrains Metaprogramming System called “Heavy Meta” on Twitch. In this blog post I would like to give you an update on my ideas for this year and look back on what worked well and what did not.

My goal with “Heavy Meta” was to do a slow paced tutorial on MPS, that is up to date with the current developments in MPS and its community. A secondary goal was to use the project build during the stream as a bases for further project ideas. So far I have produced over 20 hours of video. The good news it that I will continue to do so in 2019, but I won’t have time to do that before February. In addition to current tutorial based format I also have some ideas on what I would like to add to the streams, but more on that later.

In this post I‘m sharing some of our experiences from updating to MPS 2018.3 in various codebases. I did the migration in three projects: the MPS Extensions, mbeddr and IETS 3. The final version of MPS 2018.3 is now release for about a week.

The migration in all three projects is done. All projects compile against the lastest version of MPS and all migrations were executed. Some of the projects still contain model checking errors because of new checks introduced in the update but the generated code works. For all except one the fixes are straight forward, but some are hard to automate. This post is also not a complete list of all API changes in the tool, its meant as an overview about the different activities during them migration and mentions changes that were significant. I won’t list all the low level API change that you might encounter when you‘re building something like the MPS extensions that integrate on a very low level and depend on a lot of its internals. If you are primarily a user of MPS and building languages with it then this post should give a good impression on what has changed.

There are a number of questions around testing in JetBrains Meta Programming System (MPS) that I get a lot. For example: How does MPS affect the way I do testing? How can I do TDD in MPS? Should I test this in MPS?

In this post I will try to answer these questions, but also enable you to answer these questions for yourself. To answer these questions we have to take a step back and reflect on why we do testing in the first place. This post is not a guide on how to implement tests in MPS, it focuses on the bigger picture: what you want to test, and why.

A while back I experimented with “streaming” about MPS. I didn’t do much more than a single, not even live streamed, video. It was basically me working on some extension to MPS while I explained what I was doing which I happened to record.

Recently I revived my efforts in that direction and starte with new series in Twitch. This time really live and with recordings available on twitch afterwards. The first two “episodes” focused on a more advanced topic that was requested by members of the MPS community: the command line build and integration in a continuous integration setup. Another goal was also to see if the setup I had in mind would work technically, which seems it did.

The MPS meet-up in Munich sprung some discussions about how feature branches and language migrations get along. It seems that there is quite some uncertainty how to approach this. In this blog post I shed some light on the topic and show what we found to work in practice. I assume that you already have some background in MPS. I won’t explain the details of how to write migrations or other aspects of MPS that play a role in the overall picture.

Today I tried something new: I streamed while I was programming. Well, actually … I didn’t live stream it. I recorded it on the weekend and uploaded the unedited video file to Twitch. Mostly because my MacBook seemed to be sluggish when recording and coding. I might have solved that problem and do real live streams in the future.

In the stream I added a small feature to one of MPSs languages: MPS and other JetBrains IDEs contain a webserver that allows you to interact with the IDE from other processes. As a side project I would like to build a plugin for MPS that exposes the MPS models/programs via a REST API. In order to allow this I need access to the actual HTTP method that a request is using. At the moment MPS doesn’t expose this information to the user of their DSL which describes a handler for such HTTP requests. I decided that this might be a nice topic to stream about and so I did. You can watch the recording here:

I might be a bit late to the party since a bunch of people have already written about Slack and how awesome or awful it is. So I though I will give my two cents away how we use Slack and what I think about it

Over the past two and a half years our team has been using Slack quite heavily. Everything started relatively spontaneous as most of the team were abroad for a week and wanted to stay in touch with the others. Through these two years we have build our own understanding where slack with works and more importantly where it doesn’t.

Finally it’s there the continuation of the amazing Planet Earth documentary. I was excited when it was announced, because the first one was an amazing piece of documentary. I liked the first season (or however you want to call it) a lot but the second is taking it to a whole new level. The last documentary about nature that really got me stunned in a similar way was Wild Scandinavia (german). However Planet Earth II is taking it another step up.

While I was coding some tests for my side project I thought about if I can find a way to write my tests in a way that fits better to my way of thinking while I code.

When I write my tests my way of thinking is:

What do I want to test and what’s the expected result?

And I like it when my code and the way in which I write it reflects that. But if we have a look a usual XCTestCase we see that it’s not quite like that:

This is the first “real” post about my current side project. The first thing I will talk about is the Arduino I use to collect some date from my living room and outside of it. I will use two sensors one HDC1000 and a BME280. The BME will be placed outside because measuring the pressure in my flat does not make that much sense. Both sensors have a I²C interface so I can easily read their values from the Arduino. The Arduino I use is a simple Uno. I will be connected to my RaspberryPi via USB (more on that in the next post of the series).

This is the first post in a series of posts about my current side project. I wondered what to do with all the Arduinos, ESP8266s and a RaspberryPi that were floating around in my desks drawer. So I decided to use them to put lots of sensors in- and outside my home. I will start with the obvious ones like temperature, humidity and pressure. I also plan to add some air quality sensor, may be a VOC or CO2 sensor.

On Friday I gave a about two hours long introduction to F# to my colleagues at itemis Stuttgart. The talk starts with a overview over F# and then dives into the features of the language. It also covers some of the meta programming capability that F# offers.

Most of the code samples have been taken from the wikibook “F# Programming”.

Here are the slides:

I haven’t blogged for quite a while now. This was due two reasons, first I didn’t have that much time (read motivation) and second my old Octopress setup was horribly broken. But this weekend I choose to change this. I looked a round on the web which the new shiny blogging stuff is and stumbled up on ghost. Actually not. I know ghost for quite some time and I am running another blog which is focused on my hobby, kayaking.

In the past years I have done lots of work in C# and recently came back to work with Java because I am now working in a project that involves MPS. The Meta Programming System from JetBrains. Which is a language workbench with a projectional editor. I got quite used to some of the features C# offers. One of the features I love the most is implicit strong typed variables, expressed with the var keyword as type in a variable declarations. Due to the extensible nature of MPS it is quite easy to extend existing languages. So I decided to build this feature in MPS for Java.

Now you might think this ridiculously crazy since it is hard to change an existing language. But in MPS everything is about modularization, extensibility and composition of languages. And It ships with a Java implementation called baselanguage out of the box. For me it took 30 minutes of work to build this extension. Though I have a lot of experience with MPS and a good knowledge of baselanguage but even if you don’t have that it should take you not longer than an hour to do it on your own. And that is what this tutorial is about. What I will show here is how to extend baselanguage with a new kind of variable declaration that uses its initializer to calculate its type.

After some month of low activity in the mbeddr.arduino repository I found more time to work on it in the past week. It is now in a state that it is reasonable to release a first version of it. This version contains a very fist look at the features available by mbeddr in combination with real hardware. If you find any issues please report them on the issue tracker or get in touch with me over the various social media links on the left or via Email. Also contributions and ideas for new features are highly welcome!

Here is a little status update on the mbeddr.arduino project. We are getting closer to the first “release”, which will be an early access version but we want to get it into the hands of people to try it. The main topic on this post is not the release ;-) it is about the opportunity to avoid common pitfalls in Arduino programing with Model Driven Software Development (MDSD). This weekend I have added support for sleep modes. While I was reading the docs for avr-libc I stumbled op on the possibility that race conditions could occur when simply using the macros from avr-libc. This can happen if you just want to put the cpu to sleep if some condition is true and you are sure that the necessary interrupts to wakeup are enabled. To avoid this problem you normally write code like this:

Good news for all debian and ubuntu users using the Meta Programming System by JetBrains: I have just created a debain package for it. I have also setup a repository on this server to allow easy handling.

In order to use it open a root shell and run the following commands. Those will add the repository and import my public gpg key:

echo 'deb http://dl.logv.ws/repository ./' >> /etc/apt/sources.list
wget http://logv.ws/files/kdummann.asc
apt-key add kdummann.asc
apt-get update

After that you can simply install MPS by typing;

Recently I have published my Trainmonitor app, that is using a backend server to get the train data and do the notifications. To communicate with the server I have an HTTP API, some people might call it a RESTful API, but we will see later that this is not the case. There are several issues with the API, part of the done intentionally done wrong and others not. So why would I do stuff wrong if I knew it was wrong? Do demonstrate what’s wrong and to show what the benefit is when you do a real RESTful HTTP API with hypermedia. The idea about this came to me when I ret this blog post about HTTP APIs at XING. What the API currently is, is some kind of remote method invocation over HTTP.

This post is about setting up ant to build an mbeddr project. It enables you to do continues integration builds or other automated build scenarios. I assume that you have already installed mbeddr. If you haven’t, you should follow the instructions in the mbeddr user guide or this tutorial. It also assumes that you are using the source code distribution of mbeddr and have build it. If are using the binary distribution have a look at the end of this tutorial.

This guide covers how to build and install mbeddr from source, it does not include how to install the fancy model checking tools.

After you have installed Ubuntu 13.04 you will notice that it already ships with Java 7. Since MPS only works with Java 6 you need to downgrade it. The problem is that the official package repository of Ubuntu does not contain a Oracle Java version anymore, there is OpenJDK 6 but it will not work with properly MPS. You can go to Oracles download page and install it on your own but that is pain in the ass. Luckily there is a unofficial repository that can do all the that for you. To add the repository and install Java run this commands:

The trainmonitor / Zugmonitor Android App is now open source. Like I have planned from the beginning. Currently I only open sourceed the Android App. The server side will follow as soon as I have done some code clean ups! But it can take a while!

The App is licensed under the Apache 2 license and the source code can be found on Github.

I am developing with IntelliJ and the project has the folowing dependencies that are not part of the source code, but all open source or available from the Android SDK:

While I was working on the trainmonitor project I need a back-end server that could handle the HTTP API. The normal .Net way would be to use ASP.Net and do it with MVC, but since my back-end should run on Mono I don’t wanted to go that painful road with Mono and decided to do some stuff on my own.

That’s when logv.http was born a simple webserver written in .Net. It is not fully written by my own it is mostly a wrapper around the HttpListener class in the framework but it takes away many of the annoying work from you with a simple API and does not have the large footprint like using ASP.

I am always open to experiemt with new stuff, so this time I did something I wanted to do for a long time: play a bit with F#. I read some book about it in the past, but never had the chance to really do something with it. I was always fascinated by the expressiv code that you are able to create with it. Since my current spare time poject is using a webserver I thought it would be a nice idea to implement some stuff for it with F#. I am now rewriting the hole server component of the trainmonitor in F# to get a feeling for it. Here is the first part of code I wrote that is nothing more that a wrapper around the HttpListener of the .Net framework. I have written similar stuff in C# to get a async event based server for C#, but what I alway felt ugly about was the way the controlflow was distibuted all over the place. Wouldn’t it be nice to just write code like this:

In meiner Freizeit habe ich ein kleine Projekt gestartet das ich nun endlich in die Freiheit entlassen möchte. Es richtet sich vorallem an deutschsprachige Pendler die die Deutsche Bahn nutzen. Mehr dazu findet ihr auf einer eigenen Seite hier im Blog, auf der ich den Status des Projekts immer aktuell halten werden. Zu Trainmonitor geht es hier.

Da sich der Inhalt dieses Posts hauptsächlich an deutschsprachige Entwickler richtet verfasse ich diesen Post auf Deutsch. Es wird einer von mehreren sein, da ich dieses Thema in nacher Zukunft etwas ausführlicher bearbeiten werde. Durch Zufall bin ich bei der suche nach einer API für die Zugdaten der Deutschen Bahn auf ein sehr interessantes Projekt der Süddeutschen Zeitung (SZ) gestoßen. Es handelt sich um den Zugmonitor, der Verspätungen in nahe zu Echtzeit visualisiert und sehr erfreulicherweise auch eine API bereitstellt um mit diesen Daten zu arbeiten. Also habe ich schnell ein paar Minuten Freizeit geopfert und eine CSharp Assembly geschrieben die einen einfachen Zugriff auf die Daten ermöglicht. Das ganze ist eine Portable Class Library, die mit .Net 4, Silverlight 5 oder WinRT funktioniert. Da die Datenmenge pro Abfrage der Züge relativ groß (>1MB) sein kann habe ich ganz bewusst eine Windows Phone Kompatibilität weggelassen. Für eine Reduzierung der Datenmenge und noch einiges mehr habe ich ein anderes Projekt in der Pipeline, aber dazu mehr wenn es fertig ist ;-). Der Quellcode ist auf Github unter Apache 2 Lizenz verfügbar und die Binaries gibt es bei dort auch. Einzige Abhängigkeit ist der JSON Serialisierer von Json.Net. Bitte vergesst nicht das die API nur nicht kommerzielle Nutzung bereit gestellt wird und das die Datenquelle als “Zugmonitor von süddeutsche.de/OpenDataCity” genannt werden soll. Außerdem bittet das Team darum dass Ihr in einer kurzen Mail den Link zu eurem Projekt kundtut.

oder wie fickt man RESTful-HTTP richtig. Die Deutsche Bahn betreibt seit Jahren ein Auskunftsystem unter der Domain http://reiseauskunft.bahn.de/ dort kann man z.B. Tickets buchen, sich Verbindungen anzeigen lassen oder seit neustem auch einen Zug verfolgen. Damit kann z.B. rausfinden ob ein Zug pünktlich am Bahnhof ankommt. Eigentlich eine sinnvolle Sache, aber man kann sich keine Links auf einen Zug speichern. Denn das Webinterface der Bahn generiert immer neue URLs für die gleiche Zugnummer, diese sind auch nur eine bestimmte Zeit gültig, selbst ein einfacher refresh auf einer Ergebnisseite mit weiterführenden Links führt zu neugenerierten URLs. Da stellt sich die Frage warum das Ganze? Eigentlich würden sich die Zugnummern der Bahn perfekt eignen um einen Zug im Suchsystem adressieren zu können. Es befindet sich immer nur ein Zug mit der jeweiligen Zugnummer im Schienensystem der Deutschen Bahn. Selbst Ersatzzüge könnte man z.B. mit dem Responsecode 307 abbilden. So wäre es für jemanden der jeden Morgen den IC 2061 nutzt möglich z.B. über die URL http://reiseauskunft.bahn.de/bin/trainsearch.exe/dn/IC 2061 die aktuellen Informationen zu diesem Zug zu finden. Ein kurzer klick auf einen gesetzten Bookmark würde vollkommen ausreichen um die Informationen zu haben. Aber Dinge wie z.B. Chrome to Phone würde dann sinnvoll funktionieren. Doch bei der Bahn ist man anscheinend anderer Meinung, man möchte lieber die Menschen nerven und jedes mal in der Suchmaske den Zug eingeben lassen. Nun könnte man damit entweder zu dem Schluss kommen, dass die Bahn einfach “das Internet” nicht verstanden hat oder ein Interesse daran hat das es möglichst schwierig ist an aktuelle Zugdaten zu kommen, das eine fest URL für einen Zug es erheblich erleichtern würde die Informationen über diesen Abzugreifen. Welches davon zutreffent ist weiß ich nicht und muss jeder Leser für sich selbst entscheiden.

At first I have to put a big fat disclaimer up in here: what I am discribing below is totally undocumented and unsupported by microsoft. It is based on findings while I was investigating issues with the MemoryCache in the System.Runtime.Caching assembly. I would not recommend any useage of this in a production software. If you decide to do so that is up to you, but there is no support for this from anyone and it is likly to break in the future, since it is based on internals of the CLR. Ok lets start. All started by nailing down some issue with the MemoryCache that was allocating far more memory than the specified limits. It uses a CacheMemoryMonitor internally to monitor the amount of memory it consumes. And if it reaches its limit it will try to drop elements from the cache. To get that number it has a GetCurrentPressure method, which itself uses a class called SRef this class is internal to the assembly and has some interesting member: ApproximateSize. So I took a closer look at this class. It’s constructor takes an object and then creates an instance of a type loaded into a static field. The type is called System.SizedReference, looks we have something interesting. The type is implemented in the mscorlib and marked as internal, thats the reason why it is loaded via reflection . The class looks like this:

My current spare time project YAAF had some requirement that I need to intercept calls to a function. The interceptor then decides if the function gets called or not. But I wanted to parameters to be encapsulated, so the interceptor does not have access to them. The easiest way to do this was a simple delegate that will be passed to the interceptor and represents the complete context needed to call the callee when the interceptor decides to do so. But how to do it? With lambda expressions that is easy so I created a sample like this:

In my spare time I am working on a metro style app for geekli.st, which I wanted to write with CSharp. Since I don’t wanted to create hole REST API bindings by myself, I had a look at the official API-wrappers and  found the CSharp bindings for geekli.st by Scott Smith. But there  was a problem, there was no Windows Runtime version of it. This was not caused by the code itself, it where its dependencies. It depends on Spring.Social.Core which depends on Spring.Rest. Both of them are currently not available for the Windows Runtime. Due to the architecture of the component I decided to remove this dependency and replace it with something that would run on the Windows Runtime.

Well this is my first post on a non software development topic and I would like to say happy birthday to my favourite podcast omegatau for their 100th episode. And I would like to thank Markus and Nora for this awesome project. I think my first episode was number 22 about nuclear fission in late 2009 and felt in love with it right at that point. In the following years there were so many awesome episodes that I would recommend you to listen to each of them, but as always I have some favourites. My all time favourite German Episode is number 58 “Komplexe Systeme” which shows how to design complex systems without being complicated. Also the German episode number 65 – Eisenbahnsicherungstechnik is awesome.

In past years I was working with different aspect oriented frameworks. A lot with postsharp in its early days when it was open source and then changed its license to a more closed model. I have helped project to use aop to allow easier development and write less but expressive code that is just focused on the work it should do. I often used ready to use frameworks like postsharp or spring-aop to get that work done but those tools are pretty powerful and that might become a problem when the people in the project aren’t familiar with aop, then they start seeing nails everywhere and want to hammer them with aop. I also implemented it myself but those were focused on the project and not that genric that I would have published them to public, also the contract wouldn’t have allowed me to do so. In the last days I found some spare time to implement the basic stuff. The project should be:

The current project I did for a customer faced me with tons of legancy code which was written in C++ years back. This code is supposed to die at some point, but due some requirements the code needed to interact with some .Net / C# with COM Interop. The code uses a lot of function pointers passed to the callee to supply callbacks for logging and user notification. Exposing events for interaction from .Net via COM to the C++ code would have caused a major rewrite of the code, so we decided to stick with the callback pattern. Directly using the Marshal.GetDelegateForFunctionPointer stuff would have been possible but sadly the function does not accept generic delegate classes like Action which would have caused everyone using those functions would create his own delegate types. That was point where I decided to build a factory that creates actions from function pointers.  Looking at Expression.GetDelegateType might look promising at first but it will return Action if possible and only create a custom delegate type if it doesn’t find any matching Action. So we are forced to create our own delegates at runtime. I built a static class called NativeCallbackFactory, thathas a simple type constructor which builds ModuleBuilder for later code emititation.