ISISComputingGroup / IBEX

Top level repository for IBEX stories
5 stars 2 forks source link

Polaris Jaws: Unable to home #2742

Closed Tom-Willemsen closed 6 years ago

Tom-Willemsen commented 7 years ago

Reported by Ron Smith (Polaris)

As an instrument scientist I would like to be able to home my Galil axes reliably.

Ron came into the office and reported that homes were no longer working on one of his jaw sets.

The issue appeared to affect all motors on GALIL_02. I appear to have fixed it by restarting GALIL_02 on Polaris but we should try to understand the root cause as well.

The following IOC log messages look like they might have been the cause:

[2017-11-03 14:13:46] Home started axis F

[2017-11-03 14:14:16] sevr=major Homing timed out after 3.500000: BGF=0.000000 SCF=2.000000 

[2017-11-03 14:14:16] sevr=major F Homing timed out

[2017-11-03 14:14:16] sevr=info Poll services: MOTOR CANCEL HOME F

[2017-11-03 14:14:16] Poll services: STOP F

[2017-11-03 14:14:16] Poll services: MOTOR OFF F

[2017-11-03 14:14:16] MTR0206 HW_LIM+ 

[2017-11-03 14:19:05] Thread 1 not running on model DMC2280 Rev 1.0o-CM, address 192.168.1.202

[2017-11-03 14:19:05] 2017/11/03 14:19:05.420 asynMotorController:writeFloat64 error, status=3 axis=5, function=3, value=0.000000

[2017-11-03 14:19:05] Cannot start homing as galil threads are not running

[2017-11-03 14:20:34] Thread 1 not running on model DMC2280 Rev 1.0o-CM, address 192.168.1.202
FreddieAkeroyd commented 7 years ago

Homing is done by a separate thread in the galil, it looks like this thread had died for some reason. The threads are stated when the ioc is started, hence that would have fixed things. Thread 1 is started automatically by thread 0 when the homing routines are loaded, I don't know if there is any reason why we couldn't start thread 1 if a home is requested and it is not already running?

GDH-ISIS commented 7 years ago

The homing sequence looks as though it did start as the axis did reach the limit switch as intended. I believe it did not complete the entire sequence - it is supposed to come slowly off the limit switch afterwards. As a result, I think the homing thread was operational when the request was made to perform the homing routine. The thread should not have stopped though. This is probably my doing and I will look into whether I can fault find the homing routine. I think the suggestion by Freddie for checking whether the homing thread is running could be worth looking at. I can not think of any problems. I am not sure if this has already been touched on before in GalilAxis.cpp though (see below).

// check homing thread is available if ( !pC_->checkGalilThreads() ) { errlogPrintf("Cannot start homing as galil threads are not running"); return asynError; }

GDH-ISIS commented 7 years ago

To restart the Galil homing threads, the code may look similar to the following GalilAxis.cpp (see below). My C++ is not brilliant I'm afraid. Please note that this sequence is untested, but the principle I think is correct. To test : 1) Connect up ioc to Galil. This will start the Galil threads. 2) Issue : caput "MYPVPREFIX":MOT:DMC01:SEND_CMD_STR "AB0". This should stop the Galil threads. 3) Issue : caput "MYPVPREFIX":MOT:MTR0101.HOMF 1". Request home of an axis. 4) Look in the ioc window and confirm that there is a message saying the thread/threads are not running and trying to restart. 5) Confirm all of this using LabVIEW Galil TOM to monitor the progress of each step. Please remember this is only checking that the homing thread is alive - not the state of a homing sequence.

// check homing thread is available if ( !pC->checkGalilThreads() ) { errlogPrintf("Galil home threads are not running. Attempting to restart homing threads.\n"); sprintf(pC->cmd, "HX0;HX1"); pC->writeReadController(functionName); sprintf(pC->cmd, "XQ 0,0"); pC->writeReadController(functionName); epicsThreadSleep(.2); if ( !pC->checkGalilThreads() ) { errlogPrintf("Unable to start Galil homing threads.\n"); return asynError; } else { errlogPrintf("Galil homing threads restarted successfully.\n"); } }