generics

[christophberger]

Written on 02/25/2017 13:28:28

URL: https://appliedgo.net/generics/

[christophberger]

Migrated comment, written by Richard Eng on 03/28/2017 20:07:18

Great article! I'm using it to defend Go in my blog: https://medium.com/p/why-do...

But I'm getting a lot of pushback. This is a very divisive issue!

[christophberger]

Migrated comment, written by Christoph on 03/28/2017 20:22:47

Thanks for the feedback... and thanks for sharing the article! :-)

[christophberger]

Migrated comment, written by Christoph on 03/28/2017 20:51:52

> But I'm getting a lot of pushback. This is a very divisive issue!

Well, your article attacks the critics of Go. I am not surprised that you get pushback from them. It's human nature. But I also see some well-considered comments. Listen to these, not to the haters.

[christophberger]

Migrated comment, written by Ries van Twisk on 04/07/2017 08:47:54

After reading this all I can think of is this : Let's wait until go add's generics :)
Now, I don't know go myself but from what I am reading here is that it really doesn't make life easer and I truly believe a language should make life as easy (and reliable) as possible.

So why isn't map not a generic if you can do this? m := make(map[string]int) Isn't this not some sort of generic if you can tell the container what type it's holding?

Curious what's going to happen with this in the next 5 years...

[christophberger]

Migrated comment, written by Christoph on 04/07/2017 10:01:35

Indeed, the map type behaves like a generic data type in a couple of ways. Also, many of the built-in functions (like len(), append(), cap(), copy(), new(), etc) expose a generic-like behavior as they can operate on different types; for example, append() works with slices of any type, and the len() function works with arrays, pointers to an array, slices, strings, and channels. So Go does have some kind of partial support for "generic" behavior, although this is only a fraction of what true generics would provide.

About waiting until Go adds generics:

Why?

* If you use Go in its typical realms, chances are that you never have the need for generics.
I know of one experienced software engineer who said he barely have seen the need for generics during 25 years of software development. (Even with the languages he used before Go.)

* If you desperately need generics, then you should, by all means, look elsewhere.
But still this could be the wrong approach. Instead of asking, "does this language have generics?" better ask, "is this the right language for the task I have to solve?"

[christophberger]

Migrated comment, written by Richard Eng on 04/08/2017 13:13:56

I programmed in various languages throughout my 20-year career in IT, languages like FORTRAN, TAL, C#, Java, Python, Objective-C. My principal language was plain ol' C. In all this time, I have never needed generics, and I have never missed it. I just don't get what the big deal is. Generics is one possible language feature out of many in programming language design, and it's not even the most important nor most useful. Generics is not some kind of programming panacea.

[christophberger]

Migrated comment, written by Christoph on 04/10/2017 07:45:03

In my experience, there are a few areas of programming that benefit from generics, such as generalized data container structures (think of lists, sets, trees of type T) or numeric programming (consider matrices of int, float, bool,... and the corresponding operations). Outside of these, there seems indeed surprisingly little need for first-class language support of generics.

[christophberger]

Migrated comment, written by deepsun on 06/28/2017 22:28:36

Doesn't the last approach, "Code generation", sounds pretty much like Reification, something that C# already does internally with generics?
And "Type assertions" approach also sounds just like "type erasure", something that Java VM does internally.

[christophberger]

Migrated comment, written by Christoph on 06/30/2017 14:24:23

Indeed, other languages may uses some of the techniques discussed here internally for implementing generics. After all, the underlying problem is the same, and I think it's quite natural that the different solutions may have some overlap.

[christophberger]

Migrated comment, written by Michael Bevin on 01/15/2018 16:27:16

Still waiting till Go adds generics too ...

It doesn't make one very confident in the health of the Go community, when it keeps on coming-up with excuses+suggestions like in this article, i.e.:
- "There are already 3 (inbuilt) generic types" - well great, that shows Generics are useful+needed, at the very least for such collection types - now make generics available generally, so programmers can also use them if they're creating their own such collection types.
- "Consider Copy & Paste" - cringe
- "Use a Code Generator" - double-cringe - like copy&paste but on a larger scale - you're just working around a language deficiency with a much more cumbersome solution than the solution of properly adding the feature to the language would be. Whenever you see "Code Generator", this is a red warning flag for a lacking language feature (recalling Java days), never an ideal solution. Also, most of the 'arguments' against generics put forward here still apply to such workarounds, the end result is simply worse all around.

When will these Go language designers realise that, as awfully full of historical baggage + bad past decisions as they are, the reason languages like Javascript + C++ stay used so much, is that they provide a lot of flexibility - they don't try to over-limit and over-restrict what can be done with them, thus allowing their usage to evolve through all kinds of things being done on top of them that were never originally envisioned, rather than restricting things to the authors' limited vision of what should + shouldn't be done with the language.

[christophberger]

Migrated comment, written by Christoph on 01/15/2018 17:33:22

Thanks for your feedback Michael.

I fully understand your point regarding the status of generics in Go. Regarding this article, consider that it is already a few months old (which is a long time in Go), and the Go community has moved forward quite a bit since then. The core team is actively working towards Go 2.0, and they have requested to write experience reports to see where the language needs improvement. The community has answered with a lot of reports since then, and a large part of them are about situations where generics might have helped keeping code simpler. So there is an active discussion going on, but don't hold your breath; the process is slow because the problem is not just about adding generics, but rather adding generics the right way. There are at least a dozen different kinds of "generics", each with their specific benefits and drawbacks. Believe me, if adding generics to Go was easy, it would have been done already.

Regarding your comments on the article itself: The article does not mean to excuse anything. It just explains what the situation is and which options can be used instead of generics in specific situations. I do not claim that these options are ideal nor that they address any possible situation that other languages would use generics for. If you got a wrong impression from the article's title, I apologize. It was not meant to "defend" the lack of generics in any way. It clearly was wrong to choose a catchy title like this (yes, call it clickbaity if you want, maybe it is), and from earlier feedback I already learned that it just leads to misunderstandings, and I have taken care to choose more neutral titles since then. (I decided to keep this particular title, however, because it was already published and might have been referred to by other articles and Web pages.)

I hope you understand that I will not comment on the last paragraph. if you want to start a discussion about the design decisions the Go team made, please feel free to do so, but please keep a respectful tone towards those you criticize. Thank you.

[christophberger]

Migrated comment, written by criscola on 02/06/2018 10:28:49

Sorry but... copy & paste or code generation are simply... ugly. Reflection is a mess and not to use heavily. I would never do that if I cared about design and code organization. Interfaces are the best way IMHO, and that's ok. I also know about Go's philosophy regarding "avoid religious wars and provide the smallest number of ways to do something" and this is something I personally like a lot. Sometimes, by the way, you have to be less authoritarian and let the programmers handle their problems with their philosophy, we are just too different to conform on every single aspect. Some problems are better addressed using generics, this is undeniable. We'll see if they're going to listen to (a large chunk of) the community and add this option.

[christophberger]

Migrated comment, written by Christoph on 02/06/2018 20:50:28

I never said copy&paste is elegant :-) What you see in this article are just some workaround that i observed in the wild. I only wanted to list these workarounds without any rating or judgement, but I do agree that interfaces seem to be the next best thing when there are no generics around. And BTW, the discussion on generics in Go 2.0 is already open. (Blog post and experience reports.)

[christophberger]

Migrated comment, written by WPWoodJr on 02/13/2018 15:09:59

The reflection code doesn't seem to work. Variable g never never receives the value 3.14152 and remains 0. I can't figure out how to fix it either :)

[christophberger]

Migrated comment, written by Christoph on 02/18/2018 21:06:07

Confirmed with Go 1.10. I am quite surprised by this. I am 99.999% sure the code worked as expected when I published the article. I am curious if there is an old bug that got fixed, or a new bug introduced, since Go 1.6.
Will have to investigate further....

[christophberger]

Migrated comment, written by Chris Howie on 04/01/2018 14:55:13

I just want to comment that I didn't read any disrespectful tone from the last paragraph of Michael's comment. I believe that by "full of historical baggage + bad past decisions," Michael is referring to the experiences that led the Go designers to design Go as they did -- the mistakes they made in the past that informed their design. This is not a slight against the developers, but an acknowledgement that we learn from our mistakes.

[christophberger]

Migrated comment, written by Christoph on 04/14/2018 20:24:47

Thank you Chris for sharing your point of view. I am not a native speaker and I am thankful if someone helps sorting out a misunderstanding. However, after re-reading the paragraph in question a few times, I still cannot agree that there is no disrespect shining through these lines. What that paragraph conveys to me is:

"These Go designers are awfully full of historical baggage and bad past decisions. They over-limit and over-restrict what can be done with Go. They have limited vision."

Of course, everyone is free to have their personal opinion about the inventors of Go, and I can absolutely understand if someone disagrees with the Go authors' decisions or personalities and this is just fine. However, my concern is that the tone of this paragraph could trigger angry replies and drag the discussion down to a level that I would not want to see here. (Luckily this did not happen, so perhaps I was overreacting a bit.)

I run a little blog about a programming language here, and I love to see technical discussions evolve around the articles, where everyone can discuss as intense as they want but always with respect towards the other participants or any third parties.

[christophberger]

Migrated comment, written by RC_RC on 04/28/2018 08:54:34

As a C# developer I can't imagine writing code without generics, my code has <> everywhere. I would have to do a lot of unlearning and learning if I would start using Go. Maybe it would be very interesting experience to do that but it will take me a lot of time.

[christophberger]

Migrated comment, written by RC_RC on 04/28/2018 09:14:59

I don't think that that is what Michael Bevin writes. "as awfully full of historical baggage + bad past decisions as they are" refers to the designers of Javascript and C++. It doesn't refer to the designers of Go.

I am also not a native English speaker by the way.

[christophberger]

Migrated comment, written by Christoph on 04/28/2018 21:03:30

I see what you mean, and yes, it looks like I was wrong about that particular sentence. Thank you for helping to clarify this.

[christophberger]

Migrated comment, written by Eldan on 06/18/2018 01:05:16

It's not impossible to avoid generics but personally I feel it's a HUGE downside to using golang.
for people that don't get the importance of generics I recommend taking a look at C#' LINQ library for an example of just how powerful they can be.
Generally I think there's a lot of missing features in go that can really improve the language and make it easier to write readable code.
One is obviously generics, another is the ability to "extend" existing types.
e.g. strconv.Itoa(int(num)) can be encapsulated as num.toString()
instead of doing strings.Contains(text, subText) just do text.Contains(subText) way more elegant imho

[christophberger]

Migrated comment, written by Christoph on 06/18/2018 04:48:41

The importance of generics hugely varies with the problem domain. In some domains, like container libraries, numerics, etc, they can be a real benefit, and their absence poses a challenge to developers. In other domains, they are neither needed nor missed at all.

But what is more important is that generics, as easy as they might look to the developer who is using them, add a good deal of complexity to the type system. Moreover, there is not one notion of "generics" but rather a truckload of different approaches that all come with specific benefits and drawbacks. (This document shows how complex and difficult the task of adding generics to a type system is.)

If adding generics to a language without sacrificing other important aspects (compiler speed, type system complexity, readability, etc) was easy, the Go team would have done it already in Go 1.0.

[christophberger]

Migrated comment, written by Valentin Kovalenko on 07/17/2018 15:15:20

This article alone is enough to say a 100% no go to "Go".

[christophberger]

Migrated comment, written by Christoph on 07/18/2018 17:16:27

Looks like you are working in one of the (few) problem domains where generics can be useful. Feel free to favour another language then, Go never claimed to be a one-size-fits-all language.
(And frankly, I would deliberately stay away from any language that claims this, as it can never live up to this claim.)

[christophberger]

Migrated comment, written by Marcin on 08/04/2018 15:08:11

Hi! You c# example (`List<dictionary<string<ienumerable<httprequest>>>>`) contains an error, Dictionary in c# takes two generic parameters (it is basically a map), so I belive you wanted to write `List<dictionary<string, ienumerable<httprequest=""> >>>` (notice the comma).

[christophberger]

Migrated comment, written by Christoph on 08/11/2018 13:21:19

I am sure you are absolutely right on this, and to my defense I did not create this code snippet myself but took it from another blog post (the link to it is directly after the code snippet). I fixed the code snippet.

[christophberger]

Migrated comment, written by Tamás Barta on 08/13/2018 16:24:49

There are two sides of the "they are neither needed nor missed at all" coin. First, you can always workaround stuff. I remember how I didn't need, and didn't miss type inference in Java 7. And then I started using Kotlin, and couldn't understand, how I didn't see the benefits of type inference. Yes, one side is that if you have no access to generics, you'll still solve all your problems, but the other side is that generics could help you solve them better. Generics is the kind of feature, (as type inference is) that you only don't miss if you're not a user of it. I don't know what domains are you talking about as you'd know all the satisfied developers, but even just writing chainable APIs for interfaces is much more natural using generics, and that's not really a domain specific thing, and also I don't really know domains where such patterns may never occur. Also library authoring is one area, where you'd most likely want to use generics, since you'll never know with what types your library will be used.

If you admit generics is not added because the lack of technical feasibility, you also admit generics would make the language better in design terms. I don't care if anybody considers Go a bad or a good language, I don't use it, I don't judge it as a whole, but it's strange to read that Go developers don't miss generics, because they have all they need, in the form of code generation/reflection/type assertion/etc... It would be a much clearer statement that: "Yeah, no generics, it sucks, but here's how to workaround it.".

[christophberger]

Migrated comment, written by Ryno Kotzé on 08/22/2018 11:32:50

you mention generics but then refer to interface implementation / inheritance when saying len() function works on arrays and pointers to an array, slices, strings, ... i am unsure how that has anything to do with generics and using interfaces to replace generics is not really an option, how do you express this in an interface `class Respones[A](status: Int, data: A)`

[christophberger]

Migrated comment, written by Ryno Kotzé on 08/22/2018 11:37:06

Generics isn't for a problem domain its more a way of thinking and designing your code. You can do it without generics but that introduces a lot of rework copy paste is not a solution ever, like ever. cause you could copy and paste something now and then later on you find a bug in the original section that you copied from that means now it becomes a hunt for all the places it was pasted and you will miss 1. writing enterprise software has a big focus on reusable code and it also has a really really big focus on maintainability which is what generics offer in spades. And again its not a problem domain its a design pattern in a way.

[christophberger]

Migrated comment, written by Christoph on 08/23/2018 19:26:58

The way you think about and design your code depends quite a lot on the problem domain. Not having generics does not automatically mean that your code is mainly copypasta. Look into Go projects on GitHub and let me know how much code duplication you find. I bet it is not much, precisely because the problems they solve are very concrete and do not need the level of abstraction that generics provide.

[christophberger]

Migrated comment, written by ingun on 09/02/2018 09:15:04

Go forces me to find a way to deal with things (that I used to deal with generic on other languages) without genric, which makes me angry, and most of the time it turns out to be better in this non-generic way over time, which makes me even more angry
Still, I think Go should get generic

[christophberger]

Migrated comment, written by Christoph on 09/02/2018 14:57:57

Looks like a real dilemma! :-)

This reminds me of a comment on Reddit by /u/carsncode some time ago:

Generics are a tough one - with or without them, you're going to have problems. Not having generics makes some smart things really difficult. Having generics unfortunately makes some stupid things really easy.

BTW - check out the Go 2 draft designs that were recently posted to the Go 2 wiki - one of them is a first draft of how generics may look like in Go 2: https://go.googlesource.com...

[christophberger]

Migrated comment, written by Kouglof on 11/22/2018 15:57:46

You can't avoid generics no matter what you do, you will at best use `interface{}` like we do in C with `void*` and lose type safety or manually duplicate code. It's unbelievable Google didn't implement the generics yet.

[christophberger]

Migrated comment, written by Tom Cook on 11/29/2018 10:48:45

"Step back and revisit the requirements. Review the technical or functional specification (you should have one). Do the specs really demand the use of generics?"

This borders on imbecilic. Generic types are such a fundamental feature of modern programming languages that going without them is a constant point of friction when coding in Go, even if it's not immediately obvious that the lack of generics is the cause. The lack of generics makes a mess of various other aspects of the language.

Don't believe me? Consider: Every language worth its salt has a range-based for loop. Go has one too. It is crap compared to absolutely every other language out there because Go doesn't have generics. Actually, it's not possible to have a range-based for loop without generics, so Go has this magic operator `range` that is generic. But it's built into the language because only the language designers should be allowed to use generics, not people developing APIs in the language. The language designers are old, wise and ineffable and know how to use generics properly whereas you, the person who actually has to use their language, are a dribbling idiot who cannot be trusted with such a dangerous feature.

What's that? You want to iterate over a slice _in reverse_? Sorry. Go write an old-fashioned index-based for loop. A `reverse` function is too complicated for our language. Yes, you can write your own, in fact you can write one for every single type you want to have slices of! You want to _filter_ a slice? You've got to be kidding. Go write your own loop to do it. And so on and so on.

In what sense can we "review the technical or functional specification" and decide we never need to iterate over a slice in reverse? It's fundamental.

Rant over.

[christophberger]

Migrated comment, written by Christoph on 11/29/2018 22:17:40

At no point does the article say that Go does not need generics, let alone that it would be better off without generics. If the article creates that impression, I apologize.

I was simply starting from the status quo and collected ways of dealing with that fact. Which includes thinking about the requirements, the specification, and the resulting design of your code. When you are used to using generic types in another language and start coding in Go, you might easily become caught in repeatedly thinking, "if I only had generic types...". This, however, leads to nowhere. Rather, go with what you have and try making use of it. It is sometimes easier than it seems.

And generics are coming. The drafts are being discussed already.

[christophberger]

Migrated comment, written by denis631 on 12/01/2018 21:03:23

I just don't get how copy/paste and code-gen are acceptable solutions. In a nutshell this is what templates are doing so why not support it from the very beginning. People in order to use "custom generics" need to do all nasty things just because Rob Pike and co. think it's an "overkill". But using codegen is not ... (facepalm)

[christophberger]

Migrated comment, written by Christoph on 12/02/2018 20:16:44

The article lists workarounds, not solutions. At no point did I want to create the impression that generics are superfluous, nor that they can be easily replaced by programming techniques. Making a good generics design is hard; otherwise they would have been part of Go since Go 1.0. And since a few months, the first official generics design for Go has been released, see the Go blog, the golang-nuts Google Group, or the Go Wiki (to name just a few places) for details.

[christophberger]

Migrated comment, written by Sam on 12/19/2018 12:12:09

Generics are useful to virtually every domain either directly or indirectly.

If I am building say an application that helps businesses issue invoices, I probably won't directly ever need to create a class or an interface that is generic. However, all the math and business logic can be described very neatly using a solid collections library (that does need generics)
A solid collection library, like scala collections can turn messy, imperative programs that are 1000s of lines long into short and concise declarative programs.

Generics make very useful domain independent abstractions possible. And these abstractions are useful to most if not all domains.

That is the power of generics, and it saddens me that the hangover of the (horrible) type system of Java is preventing the designers from adding some of the most useful programming constructs to Go.
If anyone has worked with Scala / Typescript, they will immediately know what I am talking about.

[christophberger]

Migrated comment, written by Christoph on 12/22/2018 17:44:31

Agreed, generics can provide powerful abstractions, and I think there is no doubt about that part. However, for all implementations of generics that I know, this abstraction power comes at a cost, usually either runtime or compile-time performance penalties, binary bloat, and sometimes (for bad generics design) increased complexity for the developer. Here is a good discussion of existing generics approaches, including all their pros and cons.

If generics were easy to design and implement and had no drawbacks, they surely would have been part of Go since 1.0.

[christophberger]

Migrated comment, written by Sam on 12/23/2018 13:38:24

True, poorly implemented generics can make programs more complex, but there are plenty of implementations that improve the lives of developers. If this was not true, we wouldn't be seeing so many people fighting for generics.
For example, typescript balances power and speed very well. I've never noticed any performance issues in my 2 years of writing typescript.

Moreover, performance penalties of generics are minor compared to the gains you get via increased developer productivity.

Unless you are an embedded engineer, developer productivity is far more important than saving a few clock cycles at runtime / compile time.
Most web services are IO bound (read network overhead) than CPU bound so the overhead of generics is negligible, and well worth it compared to the increase in productivity you get as a developer.

I would love to see Go's designers use languages with well implemented generics and strive towards adding that to Go.

[christophberger]

Migrated comment, written by Sam on 12/23/2018 13:44:51

You have frequently argued that generics are only useful in narrow domains. I wholeheartedly disagree with that because every domain benefits tremendously from generics.
A simple example of that are collection libraries that are powered mostly though generics. Virtually every domain involves things like arrays, sets, maps and having a solid collection library makes it very straightforward to implement your business rules in that domain.

[christophberger]

Migrated comment, written by Christoph on 12/23/2018 20:31:41

May I kindly disagree. I never said generics are *only* useful in *narrow* domains. I talked about being beneficial in some domains, while not being necessary in others. Not being necessary does not mean that they might not help making a developer's life a bit more convenient. But there is a difference between being necessary and being nice to have.

[christophberger]

Migrated comment, written by Christoph on 12/23/2018 21:02:13

Developer convenience is a difficult topic to reason about. Dynamically typed languages are so convenient to work with, yet Python developers keep moving over to Go. Why? Because the added convenience at coding is paid for by fighting mysterious bugs that pop up at runtime, sometimes even days, weeks, or months after the code ran in production without any problems.
Sure, generics can be convenient for developers. But the Go team needs to work hard (and I am sure they do!) to ensure that adding generics does not sacrifice the things that Go is famous for, including the blazingly fast compiler and the ease of reading other people's code.

[christophberger]

Migrated comment, written by Adam Tang on 02/02/2019 05:12:37

Hey guys, want generics "go" others and go with non-generics programming. Then link those with rest, grpc or mq. This way you will be the best programmer.

[christophberger]

Migrated comment, written by FoxyStyle on 02/04/2019 23:27:46

I'd add it's useful for writing high level 'business' logic when you have tons of different interfaces & models but the methods that process are the same

[christophberger]

Migrated comment, written by ch007m on 02/23/2019 08:44:12

Great article. I think that there is nevertheless an issue as I expect to get the value of "3.1415" when we execute such lines "c.Get(&g) AND fmt.Printf("reflectExample: %f (%T)\n", g, g)". Maybe I missed something ?

[christophberger]

Migrated comment, written by ch007m on 02/25/2019 11:42:13

Have you been able to investigate the issue ?

[christophberger]

Migrated comment, written by Martijm on 03/04/2019 13:27:54

It's interesting how many of the 'Go needs generics' replies to this article come from developers that don't actually use Go. I like the simplicity of Go and not having to deal with generics (because you know some developers will use them all over the place whether appropriate or not). I rarely need any of the alternatives mentioned in this article.

[christophberger]

Migrated comment, written by Philip M on 03/22/2019 17:51:02

One of my issues with Go is that it's creators implemented generic types (maps, slices, channels), but then decided that the developers who use the language should not be able to do so. In other words, the 3 most core basic data types used in the Go language cannot be implemented using Go. To me this feels like a fundamental shortcoming of the language, even if turns out that the built in types are "all you need".

[christophberger]

Migrated comment, written by Ecstatic Coder on 04/10/2019 19:52:37

Nice article :)

Just a remark about the binary bloat problem. I've recently implemented my own code generator for generic types (Generis), and I must say that in practice there is absolutely NO binary bloat if you just instantiate the generic type you need, as the generated code is as simple as it can be, compared to the matching reflection/interface based code.

Actually the biggest downside is that the files which are actually compiled are not those you edit, so the error line numbers are not right. But honestly that's not a big deal IMHO, considering you can develop Go code with much more ease and conciseness...