search

ISISComputingGroup / IBEX

Top level repository for IBEX stories
5 stars 2 forks source link

HIFI: unable to load configuration #4862

Closed FreddieAkeroyd closed 4 years ago

FreddieAkeroyd commented 4 years ago

HIFI were unable to load a configuration, after some investigation it turned out that starting the cryomagnet Vi on NDXHIFI was crashing labview. Running the VI outside of SECI also lead to a crash, just before the crash it created a dialogue box with something like "Log channel access set: in:hifi:cryo:opmode:hifi – croymag – frontpanel error 9502 Left compressor". The left compressor was in error on the VI on hifi_cryomag, but was running OK. As it looked like handling the compressor error may be what was crashing labview, we rebooted ndxhifi_cryomag and that cured the compressor error and also allowed the configuration to load OK on NDXHIFI.

Have we seen this before? I believe the compressors have been in error in channel access before, but I don't recall this causing HIFI to be unable to load configurations due to labview crashing.

KathrynBaker commented 4 years ago

I saw this when I made some changes to HIFI to monitor the temperature logging, and there has been a Footprints report of HIFI having a crash and failure (45056) as well since this ticket was created.

If there are any queries about the code change I made, I can revert to the previous version without too much hassle over the shutdown, but they will lose a useful diagnostic for the quenches.

If this is related to the compressor errors, then the only other option is to update the compressors to avoid using the dll that has become more unstable – especially for the left compressor – but that is a much bigger change, and will take time.

FreddieAkeroyd commented 4 years ago

rebooted hifi cryomag to fix compressors, was able to start seci after that

KathrynBaker commented 4 years ago

There seems to be something more sinister than just a restart will fix, it was in the same state again today, restarting CRYOMAG this time actually crashed SECI when things had been running before.

Whilst I would say that this situation is regularly resolvable via various methods, I'm not sure that any single one of them is the actual fix. It may need some further alterations to the code to use CA for the extra variables rather than the external interface.