Monday, May 28, 2012

Either in Scala: A language demonstration

Some of you might have noticed that I (finally!) started learning Scala. I used it for a while now and already made a few posts about it on my german blog.
As I am still addicted to the last.fm API and I wanted to create somthing useful while learning a new programming language, I decided to start coding a brand-new all-shiny last.fm API library for scala, because nothing sufficient exists in this area. From my point of view, the best scala approach until now was made by the Guardian.

So, lets start by making a simple sample request:

class Artist(configuration:Configuration) {

  val apiService = configuration.apiService;

  def getInfo(name: String): Either[ArtistProfile, Error] = {
    val call = new Call("artist", "getInfo", Map("artist" -> name))
    val response = apiService.service(call)
    response.result(ArtistProfile.apply)
  }
}

The code looks fairly readable, doesn't it? We have a class which takes a parameter of type Configuration and a method which returns either an ArtistProfile or an Error.
Lets dig a bit into that:
The first line of getInfo creates a new call object, which stores e.g. the method to call and the methods parameters, so no magic here.
In the second line, we use the apiService field to receive the response via the service method. Nothing special is going on until now. As you can see, no type is specified for the response variable. This is one handy aspect of scala called type inference. In many cases, the Scala compiler can automatically assume the type of the variable. The val statement makes the variable immutable(say: final in Java), in contrary to a var statement. It should be sayd that you also can specify the type if you want to.
Now the response variable should contain on Object of type Response.

Lets take a look inside that class:


class Response(val xml: NodeSeq) {

  /**
   * Gets the root element of the xml inside the <lfm> tag
   */
  val dataXml = xml(0).child.collect {case e : Elem => e}
  
  /**
  * Return the Error or None
  */
  val error: Option[Error] = Error(xml)

  def result[T]( xmlConverter :(NodeSeq) => T) : Either[T, Error] =
    error.toRight(xmlConverter(xml))

}

The constructor takes an xml. In scala, all constructor parameters are stored as class fields. Again, the xml field is made immutable so noone can change it.
I don't want to go much into the details of dataXml, only that it contains the xml of the API response. The error field is far more interesting:
Error(xml) produces something of a type Option which takes Error as the type of the generic. Generics are typed between square brackets in Scala, so it is like in Java.
The option type represents either Some(Object) or None. Both Some and None are of type Option, of course. So Option[Error] means, either there is Some(Error) or None.

Now to the result method:
As you can see, it returns an Either[T, Error]. Either can be used to represent a kind of type choice. Either the API request was successful and result returns the resulting object of type T or an error occured, in which case, we return an Error object. Either has two subtypes Left and Right, which represent the left, respective the right generic of Either.
The Option type is a bit similar to Either and conveniently has two functions toLeft and toRight. You can guess that they accordingly return a Left or Right. In the example above, toRight returns an Error if the value of the error field is Some(Error).
In the None case, it returns an object of type T.

Wait! What is xmlConverter?
The answer is: A function. Scala is not only object-oriented, but also functional. If we want to parse the xml to a result object of type T, this could become a bit tedious in Java, but not in Scala.
We simply can use a function as argument, which converts the result's xml into an shiny object.
The variable name for this function argument is xmlConverter. It takes, as mentioned before, a NodeSeq (a set of xml nodes) and returns something of type T. We don't need to take care how it does that, only that we can use it.
Long story short, xmlConverter(xml) takes some xml and returns an object of type T. The created object now is  a parameter for the toRight method of the Option[Error]. If there wasn't an error, our converted object is returned and else an error.
In Java, you would create an interface(say: Lots of different interface implementations) which you then can use as a function arument, but It's not nearly as powerful as functions as argument of other functions.
At first glance it seems like magic, but if you have tried some Haskell before, it's nothing unusual anymore.
So now we have a generic result-XML to resulting object method.

Back to the artist class:

    response.result(ArtistProfile.apply)

Here we can see that the function we use here is called apply and belongs to the ArtistProfile class. It should be said that in this case, apply is static and serves as a factory method. The factory pattern is very common in Scala.

Hopefully you have now learned how beautiful an innocent-looking piece of code can be and how powerful Scala as a functional, object-oriented language is. Comparing to Java, Scala is much less verbous: Optionan braces and semi-colons, forced indentation for beautiful code and so many more genious things which aren't available in Scala.
It should be said that you can use any of your old Java code within Scala, as it runs inside the Java Virtual Machine.

Saturday, September 17, 2011

Race conditions are the result of lazy coders

Today I got a ticket about a race condition my boss identified in a self-written java data class(Java's  GreogorianCalendar class is extremely slow and just crap). The problem is very subtle and hard to reproduce.

Take a look at the following sample Java snippet:

class DateTime {
  MyCache datetime = null;

  private void initDateTime() {
    // Creates a fresh MyCache object
    // and initializes datetime.colonTime to null
    datetime = new MyCache();
  }

  String getColonTime() {
    // datetime is some kind of 
    // lazy caching variable declared 
    // somewhere(does not matter)
    if (datetime == null) {
      //Uses lazy to initlialize variable, takes some time
      initDateTime(); 
  }

  // Colon time stores hh:mm as string
  if (datetime.colonTime == null) {
    StringBuilder sb = new StringBuilder();
    //Now do some steps to build the hh:mm string
    //...
    //set colon time
    datetime.colonTime = sb.toString();
  }
  return datetime.colonTime;
  }

  public static class MyCache() {
    String colonTime = null;
    public MyCache() {}
  }
}





The problem is that the getColonTime( ) method sometimes returns null although it always should return a time in the format hh:mm, like 17:49. Your first reaction sure is "No, it will always return something, because datetime.colonTime gets the value of the StringBuilder."
You're - of course - wrong, but it is very normal that you thing this way. The little devil is sitting in the details, waiting with his triton to peak you really hard.

Assume that we have two threads A and B who call getColonTime() at the same time (or nearly, if you only got one CPU). Threads are managed by something called a scheduler, who determines which thread has to pause and which one has to be run next. There are various scheduling algorithms, but most of them act like threads were having a party and the scheduler has to divide the CPU time cake. He does by serving each thread a piece of the cake while the other threads got to wait for their own slice.
So each thread has a defines period of time it is allowed to run before being paused again by the scheduler (e.g. it ate his piece of cake and now enqueues again to get a new one). Oh and our little devil is the boss of the scheduler ;)

Back to the problem:
What happens if two threads A and B call getColonTime() at the same time? First, A enters the first if block, because datetime is null.  As it happens, the scheduler decides to pause A before it can execute initDateTime(). Now B can run, it enters the method, jumps into the first if block and creates a new MyCache instance. Notice that, as the comment says, this also will set datetime.colonTime to null.
It then resumes and executes the contents of the second if block. It's a really widely used lazy pattern - initialize the variable's value when needed once and afterwards only return that value on subsequent calls.
So, B reaches the line datetime.colonTime = sb.toString() and our colon time member now got a value and continues to just before the return statement. Our evil scheduler now might think that this might be a good opportunity to pause A and resume  B.

Thread B now calls initDateTime() the second time. The result is, guess what, a new MyCache instance is being assigned to datetime. As our intention was to initialize datetime.colonTime lazy, the value of the latter is null again. So where's the problem? It will run through the second if block again, gets the value of the StringBuilder and everything is fine again, isn't it?
Surely not. Little devil has one of its really evil days and pauses thread A before it can execute the datetime.colonTime = sb.toString() command. As a result, datetime.colonTime is still null.
Again, thread A is resumed. There's only the return statement left, so the method will return the value of datetime.colonTime, namely null.

I took a lot of efforts to write a unit test which triggers this problem at least sometimes, lets say for one in a hundred calls, but I failed. The probability for this to occur is so low that the result was null only in 5 or 6 in total of all test runs. Each test runs that method about 200 000 times using multiple threads and I did a lot of test runs....

So whats next?
I will have to make sure that I can write a reasonable test which provokes these errors. While browsing the internet, I stumbled upon a really young project called thread weaver which looks really promising Maybe I'll give it a shot on monday.

Saturday, August 20, 2011

Lifting Combo.fm to a new level with Scala

It's project week at my company! So every employee can spend one week on whatever he likes to, as long as it has benefits for the company.

I will spend this week trying to learn ScaLa. This would of course be really boring if I wouln't have a project at my hands that will be the result of this.
So I came up with Combo.fm. It is currently written in PHP, Smarty, some MySQLi and JavaScript. Especially PHP and partly Smarty can be a bit painful and aren't really contributing to the fun I normally have while programming.

The goal of the project is to evaluate IDE/Tomcat integration with Scala, getting to know the language and to try the LIFT-Framework. I am looking forward to a great programming experience and a lot of fun, fun and again - fun.
Combo.fm will be rewritten from scratch using the existing HTML and JavaScript code and hopefully extended by some features I wanted to have long ago, like authentication and a lot of other awesomeness.
Of course, the results will be published as open source code on a newly created git repository.

I will report about the next steps soon!

Monday, May 02, 2011

Twitter share button with changeable url and no external widgets.js

Yesterday I wanted to add a "share on Twitter" button to the Combo.fm homepage, so users can share their created combo stations via Twitter, The normal button available by Twitter did not work as I wanted it to, it initializes the Button on page load. If you want to change the url afterwards, nothing happens and even the iframe solution did not work. Shame on you, Twitter!

So I created a custom solution which basically can do the same, but without the counter. Best thing is, it does not depend on the widgets.js so the page doesn't take ages to load.

I basically used the link solution for the tweet button. So assume we want to share an url mydynamicurl.com/mysubpage, but use example.com as the count url(in case twitter or I get this fixed).
The example.com/mydynamicsubpageurl is created within the website dynamically, depending on the users input and you are in no way able to put it as your current url.

Put the following code somewhere in your section:

<script type="text/javascript">
function shareOnTwitter(url) {
popupWindow = window.open(url,'popUpWindow','height=300,width=500,resizable=yes,scrollbars=yes,toolbar=yes,menubar=no,location=no,directories=no,status=yes');
</script>

This function is responsible for creating a popup window where you can tweet the shared url.

For the share button use the following:

<a id="shareOnTwitter" title="Tweet example.com!" href="http://twitter.com/share?url=%%shareUrl%%&counturl=http://example.com&text=Check%20outthis%20awesome%20dynamic%20page&via=mytwitteruser" onclick="shareOnTwitter(this.href);return false;"><img alt="Tweet this!" src="http://twitter-badges.s3.amazonaws.com/t_small-a.png"></a>

The parameters are explained in the tweet button documentation linked above. If the user clicks the link, the shareOnTwitter function will be called with the link's href attribute, which will open a nice popup window that uses Twitters intent page for tweeting.
See that I use a placeholder %%shareUrl%% in the url. We will use jQuery and the JavaScript replace function to insert the actual url:
function setTwitterShareUrl(url) {
$('#shareOnTwitter').attr('href', $('#shareOnTwitter').attr('href').replace(/%%shareUrl%%/,url));
}


If you want to change the url more than one time, you might want to use the link in a template element and call the replace() function simply on the html. Then replace the old share link attribute with the new one.

Saturday, March 19, 2011

burny|net - My own personal website

The last days I spent parts of the spare time I should have spent learning maths into creating a new personal homepage, including a German blog!
You can reach it at http//www.burnynet.de/.
Don't be afraid, I will continue blogging here, too. But my mother tongue just leaves me more freedom to express what I think. A blog is like a public diary, so in consequence it is mainly for my personal record - but too for entertaining you of course :)
burny|net is not German only. I created this page mainly for the purpose to have a central place which bundles all my projects and virtual identities into one website. Plus, as you aready might have seen, Combo.fm has its home there, too :)

I hope at least my dear german readers will visit my homepage from time to time - it is work in progress and an RSS feed will come, of course, but really worth visiting. Just check out my newest blog entry there, I guess you will have a lot to laugh (even those who use Google Translate ;) ).

Tuesday, March 15, 2011

Combo.fm improves your listening experience on last.fm

History
Nearly a year ago I wrote about the radio query language and wrote a Java application called Combo.fm as a proof of concept.
Naturally, at least as far as I'm concerned, proof of concept projects are not maintained very well and buggy and the Java application was no exception. It just had some major backdraws regarding usablitly, portability and layout which made it a no-fun project.

So about two weeks ago I got the idea to re-write the whole thing from scratch using pure HTML, CSS 3, and JavasScript. I did some investigations and as I am a curious guy I tried some new stuff like jQuery, Felix Brun's Javascript Last.fm API and the 960 grid system. They are both awesome in and of themselves, but in combination they can create a quite nice look and feel.
The result was overwhelming and the project drastically improved my Javascript and CSS knowledge. No wonder Google lets their employees spend 20% of their working time on own projects - at this point they are doing something right.

The new Combo.fm
Enough talking, just see yourself:


I added last.fm's new mix radio and also the friends radio. Using cutting edge HTML 5, Combo.fm should work with all major browsers like Google Chrome, Mozialla Firefox and reqkonq.
As I am a person with lots of ideas, the current version is, lets say: A tiny step for me, but a huge one for the last.fm community. And now replace "step" with "beer" ;)

Its current features include:

  • Listen to similar artists, tags, your library, mix radio, recommendations, neighbours and the friends radio
  • Combine stations as you like by connecting them with and, or and not
  • Mainstream-ness: Set how much mainstream your music should be
  • Set song repetition rate on a per-hour basis
  • Enable or disable discovery mode
Planned:
- Sharing stations over social networks like Twitter and (maybe) Facebook
- Shout your station urls - directly from Combo.fm
- Far in the future: Last.fm player on Combo.fm which displays Combo.fm station's name properly#

Notice: Combined radio stations are still in beta stage. There might be bugs and trouble and I personally ask not to abuse it. Thank you :)

This project needs your support!

Currently the problem is that especially the new Combo.fm is largely unknown. So:
Link it! Tweet it! Like it! Share it! Put in your forums signature! Join the project! 
Spread the word! 
The Last.fm messias has arrived and it has great news to say: You can combine Last.fm's radio stations, in any order and as many as you like!

Special Thanks
To the Last.fm guys. You are doing good and reasonable work! Go on with that!
To all the last.fm moderators who tested Combo.fm and reported bugs.
Skiye for the discussions, chats and testing until the late morning.

Thank you
for using Combo.fm! It means very much to me :)

Tobias Brennecke
The creator of Combo.fm

P.S.: Can I have spam, bacon eggs and spam, but without the spam in it?

Friday, March 11, 2011

KDE: Creating a RocketDock style quick launch bar

Since a few days I've been looking for a KDE replacement of RocketDock, which is basically a quicklaunch bar at the top of the screen.
Today I came up with a nice solution, which isnt a perfect replacement, but enough for what I wanted: