search

sal55 / langs

86 stars 6 forks source link

Availability of compiler time data #10

Closed Derek-Jones closed 2 years ago

Derek-Jones commented 2 years ago

I'm interested in the data, or rather the model of performance that might best fit the data. Here is one I did earlier: https://shape-of-code.com/2019/01/29/modeling-visual-studio-c-compile-times/

Is detailed data available to be shared?

Is compile time growing quadratically or exponentially, with LOC? If a couple of intermediate LOC values were measured, a regression fit would tell us the likely rate of increase.

Measurements are needed from at least a dozen compiles of the source, to handle the impact of other processes running during compile.

sal55 commented 2 years ago

I've only just seen your comment. If you're referring to the compiler benchmark tests on this site, then do with that data as you wish.

I no longer do tests on that scale (partly because I no longer have that machine, and it would be a huge effort to get installations of all of those languages, some of which worked by luck, to redo tests on a new machine).

My interest had been to get an idea of how comfortable various installations were when given such simple stress tests. Clearly some had trouble, and could do with some attention. (They have helped with highlighting bottlenecks in my own projects.)

A few showed non-linear increase in compile-time compared with the size of the source. Some of that might be due to slow-downs because of memory getting tight. But I haven't analysed them in depth.

Derek-Jones commented 2 years ago

Ok, thanks.

If you ever encounter a decent quantity of such data, please let me know.

My book Evidence-based Software Engineering discusses what is currently known about software engineering, based on an analysis of all the publicly available data (lots of psychology) pdf+code+all data freely available.