Friday, February 20, 2015

Thoughts of a Rustacean learning Go

So as many of you may know, I really like Rust and have been programming in it for nearly a year now.

Recently, for a course I had to use Go. This was an interesting opportunity; Rust and Go have been compared a lot as the "hot new languages", and finally I'd get to see the other side of the argument.

Before I get into the experience, let me preface this by mentioning that Rust and Go don't exactly target the same audiences. Go is garbage collected and is okay with losing out on some performance for ergonomics; whereas Rust tries to keep everything as a compile time check as much as possible. This makes Rust much more useful for lower level applications.

In my specific situation, however, I was playing around with distributed systems via threads (or goroutines), so this fit perfectly into the area of applicability of both languages.


This post isn't exactly intended to be a comparison between the two. I understand that as a newbie at Go, I'll be trying to do things the wrong way and make bad conclusions off of this. My way of coding may not be the "Go way" (I'm mostly carrying over my Rust style to my Go code since I don't know better); so everything may seem like a hack to me. Please keep this in mind whilst reading the post, and feel free to let me know the "Go way" of doing the things I was stumbling with.

This is more of a sketch of my experiences with the language, specifically from the point of view of someone coming from Rust; used to the Rusty way of doing things. It might be useful as an advisory to Rustaceans thinking about trying the language out, and what to expect. Please don't take this as an attack on the language.


What I liked

Despite the performance costs, having a GC at your disposal after using Rust for very long is quite liberating. For a while my internalized borrow checker would throw red flags on me tossing around data indiscriminately, but I learned to  ignore it as far as Go code goes. I was able to quickly share state via pointers without worrying about safety, which was quite useful.

Having channels as part of the language itself was also quite ergonomic.  data <- chan and chan <- data syntax is fun to use, and whilst it's not very different from .send() and .recv() in Rust, I found it surprisingly easy to read. Initially I got confused often by which side the channel was, but after a while I got used to it. It also has an in built select block for selecting over channels (Rust has a macro).

gofmt. The Go style of coding is different from the Rust one (tabs vs spaces, how declarations look), but I continued to use the Rust style because of the muscle memory (also too lazy to change the settings in my editor). gofmt made life easy since I could just run it in a directory and it would fix everything. Eventually I was able to learn the proper style by watching my code get corrected. I'd love to see a rustfmt, in fact, this is one of the proposed Summer of Code projects under Rust!


Go is great for debugging programs with multiple threads, too. It can detect deadlocks and post traces for the threads (with metadata including what code the thread was spawned from, as well as its current state). It also posts such traces when the program crashes. These are great and saved me tons of time whilst debugging  my code (which at times had all sorts of cross interactions between more than ten goroutines in the tests). Without a green threading framework, I'm not sure how easy it will be to integrate this into Rust (for debug builds, obviously), but I'd certainly like it to be.

Go has really great green threads goroutines. They're rather efficient (I can spawn a thousand and it schedules them nicely), and easy to use.

Edit: Andrew Gallant reminded me about Go's testing support, which I'd intended to write about but forgot.

Go has really good in built support and tooling for tests (Rust does too). I enjoyed writing tests in Go quite a bit due to this.


What I didn't like


Sadly, there are a lot of things here, but bear in mind what I mentioned above about me being new to Go and not yet familiar with the "Go way" of doing things.

No enums

Rust has enums, which are basically tagged unions. Different variants can contain different types of data, so we can have, for example:
enum Shape {
    Rectangle(Point, Point),
    Circle(Point, u8),
    Triangle(Point, Point, Point)
}

and when matching/destructuring, you get type-safe access to the contents of the variant.

This is extremely useful for sending typed messages across channels. In this model. For example, in Servo we use such an enum for sending details about the progress of a fetch to the corresponding XHR object. Another such enum is used for communication between the constellation and the compositor/script.

This gives us a great degree of type safety; I can send messages with different data within them, however I can only send messages that the other end will know how to handle since they must all be of the type of the message enum.

In Go there's no obvious way to get this. On the other hand, Go has the type called interface {} which is similar to Box<Any> in Rust or Object in Java. This is a pointer to any type, with the ability to match on its type. As a Rustacean I felt incredibly dirty using this, since I expected that there would be an additional vtable overhead. Besides, this works for any type, so I can always accidentally send a message of the wrong type through a channel and it'll end up crashing the other end at runtime since it hit a default: case.

Of course, I could implement a custom interface MyMessage on the various types, but this will behave exactly like interface{} (implemented on all types) unless I add a dummy method to it, which seems hackish. This brings me to my next point:

Smart interfaces

This is something many would consider a feature in Go, but from the point of view of a Rustacean, I'm rather annoyed by this.

In Go, interfaces get implemented automatically if a type has methods of a matching signature. So an interface with no methods is equivalent to interface{}; and will be implemented on all types automatically. This means that we can't define "marker traits" like in Rust that add a simple layer of type safety over methods. It also means that interfaces can only be used to talk of code level behavior, not higher level abstractions. For example, in Rust we have the Eq trait, which uses the same method as PartialEq for equality (eq(&self, &other)), and the behavior of that method is exactly the same, however the two traits mean fundamentally different things: A type implementing PartialEq has a normal equivalence relation, whilst one that also implements Eq has a full equivalence relation. From the point of view of the code, there's no difference between their behavior. But as a programmer, I can now write code that only accepts types with a full equivalence relation, and exploit that guarantee to optimize my code.

Again, having interfaces be autoimplemented on the basis of the method signature is a rather ergonomic feature in my opinion and it reduces boilerplate. It's just not what I'm used to and it restricts me from writing certain types of code.

Packages and imports

Go puts severe restrictions on where I can put my files. All files in a folder are namespaced into the same package (if you define multiple packages in one folder it errors out). There's no way to specify portable relative paths for importing packages either. To use a package defined in an adjacent folder, I had to do this, whereas in Rust (well, Cargo), it is easy to specify relative paths to packages (crates) like so. The import also only worked if I was developing from within my $GOPATH, so my code now resides within $GOPATH/src/github.com/Manishearth/cs733/; and I can't easily work on it elsewhere without pushing and running go get everytime.

Rust's module system does take hints from the file structure, and it can get confusing, however the behavior can be nearly arbitrarily overridden if necessary (you can even do scary things like this).
 

Documentation

Rust's libraries aren't yet well documented, agreed. But this is mostly because the libraries are still in flux and will be documented once they settle. We even have the awesome Steve Klabnik working on improving our documentation everywhere. And in general caveats are mentioned where important, even in unstable libraries.

Go, on the other hand, has stable libraries, yet the documentation seems skimpy at places. For example, for the methods which read till a delimiter in bufio, it was rather confusing if they only return what has been buffered till the call, or block until the delimiter is found. Similarly, when it comes to I/O, the blocking/non-blocking behavior really should be explicit; similar to what Sender and Receiver do in their documentation.

Generics

This is a rather common gripe -- Go doesn't have any generics aside from its builtins (chans, arrays, slices, and maps can be strongly typed). Like my other points about enums and interfaces, we lose out on the ability for advanced type safety here.

Overall it seems like Go doesn't really aim for type safe abstractions, preferring runtime matching of types. That's a valid choice to make, though from a Rust background I'm not so fond of it.

Visibility

Visibility (public/private) is done via the capitalization of the field, method, or type name. This sort of restriction doesn't hinder usability, but it's quite annoying.

On the other hand, Rust has a keyword for exporting things, and whilst it has style recommendations for the capitalization of variable names (for good reason -- you don't want to accidentally replace an enum variant with a wildcard-esque binding in a match, for example), it doesn't error on them or change the semantics in any way, just emits a warning. On the other hand, in Go the item suddenly becomes private.


Conclusion

A major recurring point is that Go seems to advocate runtime over compile time checking, something which is totally opposite to what Rust does. This is not just visible in library/language features (like the GC), but also in the tools that are provided — as mentioned above, Go does not give good tools for creating type safe abstractions, and the programmer must add dynamic matching over types to overcome this. This is similar (though not the same) as what languages like Python and Javascript advocate, however these aren't generally interpreted, not compiled (and come with the benefits of being interpreted), so there's a good tradeoff.


Go isn't a language I intend to use for personal projects in the near future. I liked it, but there is an overhead (time) to learning the "Go way" of doing things, and I'd prefer to use languages I already am familiar with for this. This isn't a fault of the language, it's just that I'm coming from a different paradigm and would rather not spend the time adjusting, especially since I already know languages which work equally well (or better) to its various fields of application.

I highly suggest you at least try it once, though!



Wednesday, February 18, 2015

Superfish, and why you should be worried

I'll be updating this post as I get more information .

For non-techies, skip to the last paragraph of the post for instructions on how to get rid of this adware.

Today a rather severe vulnerability on certain Lenovo laptops was discovered.


Software (well, adware) called "Superfish" came preinstalled on some of their machines (it seems like it's the Yoga series).


The software ostensibly scans images of products on the web to provide the user with alternative (perhaps cheaper) offers. Sounds like a mix of annoying and slightly useful, right?


Except to achieve this, they do something extremely unsafe. They install a new root CA certificate into the Windows certificate store. The software works via a local proxy server which does a man in the middle attack on your web requests. Of course, most websites of importance these days use HTTPS, so to be able to successfully decrypt and inject content on these, the proxy needs to have the ability to issue arbitrary certificates. It's a single certificate as far as we can tell (you can find it here in plaintext form)

It also leaves the certificate in the system even if the user does not accept the terms of use.

This means that a nontrivial portion of the population has a untrusted certificate on their machines.

It's actually worse than that, because to execute the MITM attack, the proxy server should have the private key for that certificate.

So a nontrivial portion of the population has the private key to said untrusted certificate. Anyone owning one of these laptops who has some reverse engineering skills has the ability to intercept, modify, and duplicate the connections of anyone else owning one of these laptops. Bank logins, email, credit card numbers, everything. One usually needs physical proximity or control of a network to do this right, but it's quite feasible that this key could be sold to someone who has this level of access (e.g. a secretly evil ISP). Update: The key is now publicly known

This is really bad.

Installing random crapware on laptops is pretty much the norm now; and that's not the issue. Installing crapware which causes a huge security vulnerability? No thanks. What's especially annoying is their attitude towards it; they haven't even acknowledged the security hole they've caused.

The EFF has a rather nice article on this here.

As far as we can tell, Firefox isn't affected by this since it maintains its own root CA store, however we are still trying to verify this, and will see if we can block it in case Firefox is affected, for example, if Superfish installs the cert in Firefox as well. If you have an affected system and can provide information about this, feel free to comment on that bug or tweet at @ManishEarth.

The application seems to detect Firefox and install some add ons as well as the certificate. We're looking into this, further insights would be valuable.

Superfish does NOT infect Firefox.

Chrome uses the OS's root CA store, so it is affected. So is IE.

 It turns out that internally they're using something called Komodia to generate the MITM; and Komodia uses a similar (broken) framework everywhere -- the private key is bundled with it; and the password is "komodia".

If you have friends at Microsoft who can look into this, please see if a hotfix can be pushed, blacklisting the certificate. Whilst it is possible for an "arms race" to happen between Superfish and Windows, it's unlikely since there's more scrutiny now and it would just end up creating more trouble for Superfish. The main concern right now is that rogue root CA cert is installed on many laptops, and the privkey is out there too. 


Update: Microsoft has pulled the program and certificates via Windows Defender! Yay! Firefox is probably going to follow suit -- now that the program itself should be gone, blacklisting the certificate won't make infected users have unusable browsers.

If you own a Lenovo laptop that came preinstalled with Windows (especially one of these models), please check in your task manager for an app called "VisualDiscovery" or "Superfish". Here's a small guide on how to do this. It's slightly outdated, but the section on uninstalling the program itself should work. Then, follow the steps here to remove all certificates with the name "Superfish" from the root store. Then go and change all your passwords;  and check your bank history amongst other things (/email/paypal/etc)  for any suspicious transactions. Chances are that you haven't been targeted, but it's good to be sure.

Alternatively, open Windows Defender, update and scan. There are more methods here