ISISComputingGroup / IBEX

Top level repository for IBEX stories
5 stars 2 forks source link

Datastreaming: get filewriter running for all instruments #5778

Open rerpha opened 4 years ago

rerpha commented 4 years ago

Currently the filewriter is only able to run for one instrument as the runInfo messages are pushed to per-instrument topics on kafka. to get it working for all instruments requires amalgamating all config messages into one topic (could be called ALL_runInfo)

Acceptance criteria

JamesKingWork commented 3 years ago

I'm guessing this has a points of 0 as it is datastreaming?

rerpha commented 3 years ago

I will update the acceptance criteria as the stream processor did not work successfully in production (more info here) and this will now be done as part of the script that fills in the missing data into the filewriter config.

rerpha commented 3 years ago

We are going to shelve this for now as some of the metadata isn't easy to set up in the file for all instruments. the main obstacle is that the histogram streams have to have their spectrum map IDs specified and this is different for each instrument.