food for thought

[martenson]

• Formally defining what the best input format for current and future analysis would look like.
• Related to the point above: exploring @erasche's GRT - which is a tool for extracting and gathering data from Galaxy: https://github.com/erasche/galactic-radio-telescope https://github.com/galaxyproject/galaxy/pull/2455 https://github.com/galaxyproject/galaxy/pull/4376
• Think about how to approach a situation where you have runtime data from 20 galaxies that could be very non-homogeneous. Does it matter? Can we mix them?
• @jmchilton's point from the RFC

  Going through all the most predictive indicators and compare them to GRT's collection process - make sure that GRT is capturing everything we need.

• Explore another John's point

  If we were simply adding a little noise in our submission request parameters (core count mostly) and recording the memory jobs consumed - this would be hugely useful is analyzing our CPU usage, based on the preliminary data it seemed like we are allocating too many cores for multiple tools. Better allocation could speed up everyones jobs.

• ^ What params we could capture on job submission that would help us the most with our time/memoru/cpu estimates?

[hexylena]

Yeah GRT's entire purpose from day 1 was capturing all of the information needed for analysis exactly like this. So if there's data that's missing from it, let's add it.

[atyryshkina]

Some extra information we might want to capture with GRT:

• memory
• number of other jobs running on the node when the job starts
• number of jobs in the queue when the job is submitted ( so jobs lined up in front of it - another number to tell us how busy the server is at the time)
• number of jobs in the queue when the job starts running ( so jobs lined up behind it)

I can't think of any other information to capture right now. Would we be able to get this information with the GRT?

[martenson]

@atyryshkina Not at the moment, but we are capable of changing the capture process to allow this I believe.

[atyryshkina]

Another attribute to track:

• cpu time