GSOC 2018

[bararchy]

Hi All, After talking with a few members of the community we thought it would be a cool idea to nominate Crystal for GSOC2018 .

We need some ideas on projects and a list of people who are cool with mentoring . We have until the 23 of Jan to finish: https://developers.google.com/open-source/gsoc/timeline

I'll update the issue and add anyone willing

Project Ideas

Crystal

• Compiler
• std libs (specifically)
• Compiler from scratch
• Polish windows support
• CI\Automation

Shards

• Kemal
• Amber
• Lucky
• crystal-db
• shards missing

Mentors

• @bararchy
• @sdogruyol
• @asterite
• @straight-shoota
• @bcardiff
• @oprypin

Students

• @MakeNowJust
• @RX14
• @bew

[sdogruyol]

Not sure how GSOC process works but I think the ecosystem can benefit from the improvements and widely used shards. Especially crystal-db, since it's the defacto shard for DBs and already used by many ORMs e.g.

If not we can apply for major web frameworks like Amber, Lucky, Kemal e.g since they are mostly used by the community.

[bararchy]

@sdogruyol Looking at the details needed, it looks like suggesting multiple projects is better then a single one, this will let students chose what interest them most

[bew]

Was thinking, in addition to improve existing shards, maybe we could allow new shards on needed subjects from https://github.com/crystal-community/crystal-libraries-needed/issues ?

And let's not forget Crystal itself (compiler, or simpler: the stdlib) (thanks @RX14 for pointing it out)

[makenowjust]

I'd like to join GSOC to improve Crystal compiler. However... who will become my mentor? (In other words, who knows about Crystal compiler better than me?)

[asterite]

I know the Compiler, but to be honest many parts of it are a mess that I don't know how to change or fail to remember how they work (well, some parts are complex).

I personally think rewriting the compiler from scratch (just the semantic analysis phase plus codegen (the lexer, parser, and macro interpreter are probably fine) is better instead of fixing things here and there. But I could probably try to mentor someone for this fixes, even if I don't think it might be worth it in the long run.

[bew]

@MakeNowJust For compiler related stuff I think @asterite or people working at Manas would be a good fit, but I don't know if they would be willing to be a mentor.

[bararchy]

It seems @asterite is willing (Thanks!) also, maybe we can get @waj or @matiasgarciaisaia to help with that?

[bararchy]

@asterite Can you detail some specific parts of the Compiler which you think should get some love? this will help in writing the application to GSOC (having most data etc..)

[asterite]

Mmm... as I said, I think it needs a rewrite from scratch. Maybe we could fix some bugs regarding generic types, or modules, and other things. Basically every bug listed here. But I think most of these can't be fixed easily without rethinking how things are modelled (generic types and modules are done in a very ugly way, or, well, as the language evolved they kind of fell short).

I don't know how to approach the compiler anymore. I tried doing some refactors here in and there in the past to improve things or to better model things, but most of it is all tied together that it's hard to change something without having to change something else, or without something breaking.

Plus, it makes no sense to fix and improve things without having a clear idea of how the language should work (and be implemented) regarding modules, generic types, etc.

But maybe it can be done, I don't know. You could try to fix a bug, I can explain how the compiler more or less works regarding that part, and the coder can suggest and try new ideas instead of just fixing things by patching stuff.

[makenowjust]

I absolutely agree with @asterite's opinion, that it needs a rewrite compiler from a scratch. I feel excited to rewrite a compiler and to help Ary.

[asterite]

By the way, and this is just a personal opinion or intuition solely based on what I think I know and from the last commits in the few years, I don't think anyone else knows how the compiler works. There are some presentations and docs about the type inference and type flow algorithms, and @waj and @bcardiff probably know how methods are instantiated and typed, but I don't remember them seeing how classes, modules and generic types are implemented, for example. Then many parts of the codegen are also a mistery to others.

The last times I tried to make significant changes in the compiler, I failed: rough edges, segfaults, wrong codegen, etc. I got a bit frustrated. Plus changing these things case by case without thinking well how they should behave in all cases is not good, in my opinion.

Maybe if the community has time, they should try to define how the language should work and slowly start implementing it from scratch. After all, there's no rush: a year from now I don't think there will be significant changes in the language, maybe parallelism will be implemented but that doesn't affect the language, just the std/runtime. Or maybe some big bug fixes will happen. But if after all that time a separate compiler with a much better organization and documentation also happens, and passes all specs, then we could make the switch.

[bararchy]

@asterite @MakeNowJust Added "compiler from scratch" to optional projects.

[marksiemers]

@asterite and @MakeNowJust I don't pretend to know anything about the crystal compiler, but I want to caution against the temptation of a rewrite.

Rewrites have killed many software products and companies in the past, and I don't know of an example of a "successful" rewrite.

https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/

It killed Netscape, most of Borland's products, and it may kill Bootstrap.

There are plenty of reasons to want a rewrite but it comes with two major costs:

1. Throwing out thousands to millions of hours of people beating on the code to uncover bugs.
2. Spending hundreds or thousands of hours coding and adding no new features.

[asterite]

Crystal is not a company. Plus the rewrite can happen in parallel. And what about YARV? This was a successful major rewrite in Ruby.

[RX14]

We do happen to have a fairly robust test suite which helps a lot. Plus, as @asterite said, crystal isn't a company so it can't really "die" per se. I'd warn against having more than two people working on a rewrite though, and would want to see it developed fairly in-tree and in-parallel, instead of becoming a long-running branch which exhibited eldritch merges with the core.

[marksiemers]

@asterite It's a good point about YARV being successful. And Crystal is essentially your language, so of course, do whatever you think is best with it.

Using YARV as an example, it started about 10 years after Ruby's initial release - meaning the language's API was pretty stable from years of use. It also took at least 2 years before it was merged, and an additional year before it became part of the official release.

I think your time spent on Crystal is particularly valuable. Speaking from a "customer" of Crystal standpoint. I would want to see a 1.0 version, windows support, and more performant db drivers as a priority over a compiler and/or language re-write that may take a year or longer.

@RX14 - The robust test suite should help a lot, but before a 1.0, the API and test suite aren't necessarily stable. A rewrite becomes a bit of a moving target.

A rewrite will no doubt benefit the language in the long run. Just please consider the costs and risks as well, and how long it may take to incorporate the rewrite.

[bararchy]

We now probably need someone from Manas or Core-Dev to take the role of Org-Admin, it can be one of the Mentors, it can't be one of the students from obvious reasons. In the guide, it is said to prefer someone who isn't mentor or student.

@waj , @matiasgarciaisaia, @mgarciaisaia , @ggiraldez, @ysbaddaden

[asterite]

Small note: @ysbaddaden doesn't work at Manas.

[bararchy]

@asterite True, my bad.

@ysbaddaden, would you be ok to Mentor? @oprypin, would you be ok to Mentor?

Is there a need for automation\infrastructure tasks? (CI,releases, etc..?)

[oprypin]

I would be OK to mentor.

---

When registering the organization, take note of this:

If you are a new organization to GSoC, is there a Google employee or previously participating organization who will vouch for you? If so, please enter their name, contact email, and relationship to your organization.

I can be that contact. Please don't comment about this in this thread, instead contact me directly for any questions.

[drhuffman12]

In reguards to the new 'from scratch' compiler:

• Would it make sense to implement the new 'from scratch' compiler as a mix of [new] shards?
• Which of these shards could be implemented independantly/sequentially/etc?
• Would any of these shards be useful elsewhere?

[drhuffman12]

Or, instead of a total-re-write all at once, could the current compiler be split into shards ... and then these shards could be re-worked as time/resources permit?

[straight-shoota]

The compiler can use external tools like a proper markdown parser for example. But that's not the point, we can do such things already quite easily as the interface is quite small. The discussion about a rewrite is about changing the compiler's very core, the internal architecture. Untangle the evolutionary developed mingle-mangle, make it better understandable and allow easier feature additions. That can't really be split into individual pieces (let alone shards) because it's all interconnected. You change something, it'll require changes in 10 other places and you quickly get around having to change everything.

[j8r]

By the way, the split out of non essentials compiler's elements will save time to core-devs. They will be able to delegate more work like code, reviews and support, and thus to concentrate more on the core compiler itself, where they are the kings.

I'm very excited about this perspective.

[j8r]

On the GSOC side, this will become easier for students to contribute on components they choose than on a monolitic project, like @bararchy said.

[RX14]

So, this got a lot of traction but based on the timeline it seems we only have 10 days to get an application for the organisation in. I think the org admin should be from manas, personally, but mentors can be from outside. Not entirely sure of next steps I just wanted to push this on and make sure we don't forget.

[bararchy]

@RX14 I'm waiting for Manas guys as you suggested , let's hope they have 10 min sometime soon to take a look at this

[konovod]

IDE plugin/integration is imho another possible topic - it doesn't require deep knowledge of compiler, but still some understanding of language parser and types system.

[matiasgarciaisaia]

Hi everyone!

Thanks for the interest & support showed in this issue.

We're working on the application to the GSoC. We'll keep you posted in this issue about the status of the application.

You can preview the would-be-official GSoC page on the site, should we get accepted: https://crystal-lang.org/summerofcode/

If you want to suggest a new project idea, or offer as a mentor, feel free to comment here or reach out through any of the usual channels.

Cheers!

[drhuffman12]

Looks great! I assume you plan to add https://crystal-lang.org/summerofcode/ to the main menu?

[matiasgarciaisaia]

Hello everyone!

Just wanted to let you know that we have already applied to GSoC - the next step in the Timeline is waiting until 12-FEB, when Google says which orgs have been accepted and which don't.

Cross your fingers, or whatever good-luck-thingy you believe in :)

[Sija]

I'd suggest changing path from /summerofcode to /gsoc2018 since (hopefully) there will be more than one instance of Crystal being part of it.

[oprypin]

https://summerofcode.withgoogle.com/organizations/

Crystal is not on the list.

[mgarciaisaia]

We've just received an email from Google saying that we didn't make it into GSoC this year.

Thanks for all the interest, and hope we have better luck next year!

[bew]

@mgarciaisaia Is there additional information as to why?

[matiasgarciaisaia]

@bew Nothing specific, really:

Google Summer of Code

Thank you for applying to be a Google Summer of Code 2018 mentor organization. Sadly, we were unable to accept Crystal Language this year. We had many more applications than available slots. We hope you will apply again in the future!

It has always been one of the possibilities - we'll try to be better prepared next year, try and get more traction/publicity, better define some tasks to suggest, but we always have the chance of Google saying "Sorry but no" ¯\_(ツ)_/¯