search

epics-modules / autosave

APS BCDA synApps module: autosave
https://epics-modules.github.io/autosave/
Other
8 stars 33 forks source link

Autosave when saving a string without value causes fprintf ERRLEVEL #55

Open tim-speight opened 1 year ago

tim-speight commented 1 year ago

Autosave when saving a string without value causes fprintf ERRLEVEL, which is problematic as you will incur data loss at the first null value one is trying to autosave.

Can null values be checked for and make this a warning or blank for the output sav files and the datestamped sav files

MarkRivers commented 1 year ago

@tim-speight can you provide an example? I don't seem to see a problem when autosaving string PVs that are empty. For example this is the value in the auto_settings.sav file for the .DESC field of an ai record which has no value:

13IDC:A1init.DESC

This is what caget and cainfo show for that PV:

corvette:iocBoot/ioc13idc_linux/autosave>caget 13IDC:A1init.DESC
13IDC:A1init.DESC
corvette:iocBoot/ioc13idc_linux/autosave>cainfo 13IDC:A1init.DESC
13IDC:A1init.DESC
    State:            connected
    Host:             164.54.160.146:5064
    Access:           read, write
    Native data type: DBF_STRING
    Request type:     DBR_STRING
    Element count:    1

This is what dbpr shows for that record in the IOC:

epics> dbpr 13IDC:A1init 2
ACKS: NO_ALARM      ACKT: YES           AMSG:               ASG :
BKPT: 00            DESC:               DISA: 0             DISP: 0
DISS: NO_ALARM      DISV: 1             DLY0: 0             DLY1: 0.02
DLY2: 0.02          DLY3: 0.02          DLY4: 0.02          DLY5: 0.02
DLY6: 0.02          DLY7: 0.02          DLY8: 0.02          DLY9: 0.02
DLYA: 0             DLYB: 0             DLYC: 0             DLYD: 0
DLYE: 0             DLYF: 0             DO0 : 0             DO1 : 0
DO2 : 0             DO3 : 0             DO4 : 0             DO5 : 0
DO6 : 0             DO7 : 0             DO8 : 0             DO9 : 0
DOA : 0             DOB : 0             DOC : 0             DOD : 0
DOE : 0             DOF : 0             DOL0: CONSTANT
DOL1: DB_LINK 13IDC:A1invert_on.VAL NPP NMS
DOL2: DB_LINK 13IDC:A1blank_on.VAL NPP NMS
DOL3: DB_LINK 13IDC:A1sens_num.VAL NPP NMS
DOL4: DB_LINK 13IDC:A1gain_mode.VAL NPP NMS                 DOL5: CONSTANT
DOL6: CONSTANT      DOL7: DB_LINK 13IDC:A1bias_on.VAL NPP NMS
DOL8: DB_LINK 13IDC:A1bias_tweak.VAL NPP NMS
DOL9: DB_LINK 13IDC:A1filter_type.VAL NPP NMS               DOLA: CONSTANT
DOLB: CONSTANT      DOLC: CONSTANT      DOLD: CONSTANT      DOLE: CONSTANT
DOLF: CONSTANT      DTYP:               EVNT:
FLNK: DB_LINK 13IDC:A1init1.PROC        LCNT: 0             LNK0: CONSTANT
LNK1: DB_LINK 13IDC:A1invert_on.VAL PP MS
LNK2: DB_LINK 13IDC:A1blank_on.VAL PP MS
LNK3: DB_LINK 13IDC:A1sens_num.VAL PP MS
LNK4: DB_LINK 13IDC:A1gain_mode.VAL PP MS                   LNK5: CONSTANT
LNK6: CONSTANT      LNK7: DB_LINK 13IDC:A1bias_on.VAL PP MS
LNK8: DB_LINK 13IDC:A1bias_tweak.VAL PP MS
LNK9: DB_LINK 13IDC:A1filter_type.VAL PP MS                 LNKA: CONSTANT
LNKB: CONSTANT      LNKC: CONSTANT      LNKD: CONSTANT      LNKE: CONSTANT
LNKF: CONSTANT      NAME: 13IDC:A1init  NAMSG:              NSEV: NO_ALARM
NSTA: NO_ALARM      OFFS: 0             PACT: 0             PHAS: 0
PINI: NO            PREC: 0             PRIO: LOW           PUTF: 0
RPRO: 0             SCAN: Passive       SDIS: CONSTANT      SELL: CONSTANT
SELM: All           SELN: 1             SEVR: INVALID       SHFT: -1
STAT: UDF           TIME: <undefined>   TPRO: 0             TSE : 0
TSEL: CONSTANT      UDF : 1             UDFS: INVALID       VAL : 0

What version of autosave are you using?

tim-speight commented 1 year ago

Hi Mark, I am using 5.10.2 and doing some config for EPICS MOTOR with the motorAcsMotion controllers. ile "$(TOP)/db/basic_asyn_motor_clf.db" { pattern {N, M, DTYP, PORT, ADDR, DESC, EGU, TWV, SMGR, DIR, VELO, VMAX, VBAS, ACCL, BDST, BVEL, BACC, HLSV, HVEL, JVEL, JAR, SREV, MRES, ERES, PREC, DHLM, DLLM, URIP, UEIP, STAGE_TYPE} {1, "MS-$(N)", "asynMotor", ACS1, 0, "axis$(N)", "°", 1, 0.02, Pos, 1, 20, .1, .2, 0, 1, .2, 0, 1, 10, 10, 200, 1, 1, 0, 0, 0, 1, 0, ""}

tim-speight commented 1 year ago

Record describing the loop status

record(stringout, "$(P)$(M):STAGE_TYPE") { field(DESC, "Stage or Load type") field(VAL, "$(STAGE_TYPE=0)") }

When I forgot to get my substitution file prepared - it ended up with a null value in the system and at the point it tried to write my new additional m1:STAGE_TYPE field autosave came up with cannot write file at its dated and time 'sav file'...

tim-speight commented 1 year ago

FYI I'm writing additional values in like "Lin OLV" - for a linear stage with OpenLoop verfication. And Rot OL for plain old open loop rotational stage and Lin ACL - to deign it has an active closed loop for that linear stage. More of an engineers description as I find EPICS MOTOR record a bit lacking here...

MarkRivers commented 1 year ago

Record describing the loop status

record(stringout, "$(P)$(M):STAGE_TYPE") { field(DESC, "Stage or Load type") field(VAL, "$(STAGE_TYPE=0)") } When I forgot to get my substitution file prepared - it ended up with a null value in the system and at the point it tried to write my new additional m1:STAGE_TYPE field autosave came up with cannot write file at its dated and time 'sav file'...

I am not sure I understand this. A zero-length string has a null character in the first byte, and zero-length strings are saved with autosave with no problem. I have not tested what happens when the macro is not present like your case, but it seems strange that would not work.

tim-speight commented 1 year ago

I will try to give a better worked example. All my Linux perms were fine the file actually wrote the vast majority of the record and when I took a look at the dated sav file it blew up on that last field I had recently added to my UAT system

Sent from Android- 07725 477807

From: Mark Rivers @.> Sent: Friday, July 7, 2023 6:09:59 PM To: epics-modules/autosave @.> Cc: Speight, Timothy (STFC,RAL,CLF) @.>; Mention @.> Subject: Re: [epics-modules/autosave] Autosave when saving a string without value causes fprintf ERRLEVEL (Issue #55)

Record describing the loop status

record(stringout, "$(P)$(M):STAGE_TYPE") { field(DESC, "Stage or Load type") field(VAL, "$(STAGE_TYPE=0)") } When I forgot to get my substitution file prepared - it ended up with a null value in the system and at the point it tried to write my new additional m1:STAGE_TYPE field autosave came up with cannot write file at its dated and time 'sav file'...

I am not sure I understand this. A zero-length string has a null character in the first byte, and zero-length strings are saved with autosave with no problem. I have not tested what happens when the macro is not present like your case, but it seems strange that would not work.

— Reply to this email directly, view it on GitHubhttps://github.com/epics-modules/autosave/issues/55#issuecomment-1625703461, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOQ5OMRPF73IC4CAI62B5K3XPA7GPANCNFSM6AAAAAA2BTAGOU. You are receiving this because you were mentioned.Message ID: @.***>

MarkRivers commented 1 year ago

I just tried to reproduce the problem and I cannot.

Here is my startup script:

corvette:CARS/iocBoot/ioc13test>more st.cmd.string_test
< envPaths
dbLoadDatabase("../../dbd/CARSLinux.dbd")
CARSLinux_registerRecordDeviceDriver(pdbbase)
dbLoadRecords("string_test.db", "")
< ../save_restore_IOCSH.cmd
iocInit
create_monitor_set("auto_settings.req", 30)

Here is string_test.db:

corvette:CARS/iocBoot/ioc13test>more string_test.db
record(stringout, "string_test") {
field(DESC, "This is the description")
field(VAL, "$(STAGE_TYPE=0)")
}

This is the complete output when the IOC boots:

corvette:CARS/iocBoot/ioc13test>../../bin/linux-x86_64/CARSApp st.cmd.string_test
< envPaths
epicsEnvSet("IOC","ioc13test")
epicsEnvSet("TOP","/home/epics/support/CARS")
epicsEnvSet("SUPPORT","/corvette/home/epics/support")
epicsEnvSet("AREA_DETECTOR","/corvette/home/epics/support/areaDetector-3-12-1")
epicsEnvSet("ASYN","/corvette/home/epics/support/asyn-4-44-2")
epicsEnvSet("ADSUPPORT","/corvette/home/epics/support/areaDetector-3-12-1/ADSupport")
epicsEnvSet("ADCORE","/corvette/home/epics/support/areaDetector-3-12-1/ADCore")
epicsEnvSet("AUTOSAVE","/corvette/home/epics/support/autosave-5-10-2")
epicsEnvSet("BUSY","/corvette/home/epics/support/busy-1-7-3")
epicsEnvSet("CALC","/corvette/home/epics/support/calc-3-7-4")
epicsEnvSet("SNCSEQ","/corvette/home/epics/support/seq-2-2-9")
epicsEnvSet("SSCAN","/corvette/home/epics/support/sscan-2-11-5")
epicsEnvSet("DEVIOCSTATS","/corvette/home/epics/support/devIocStats-3-1-16")
epicsEnvSet("EPICS_BASE","/corvette/usr/local/epics/base-7.0.7")
epicsEnvSet("ADANDOR","/corvette/home/epics/support/areaDetector-3-12-1/ADAndor")
epicsEnvSet("ADROPER","/corvette/home/epics/support/areaDetector-3-12-1/ADRoper")
epicsEnvSet("ADLIGHTFIELD","/corvette/home/epics/support/areaDetector-3-12-1/ADLightField")
epicsEnvSet("ADMAR345","/corvette/home/epics/support/areaDetector-3-12-1/ADmar345")
epicsEnvSet("ADMARCCD","/corvette/home/epics/support/areaDetector-3-12-1/ADmarCCD")
epicsEnvSet("ADPERKINELMER","/corvette/home/epics/support/areaDetector-3-12-1/ADPerkinElmer")
epicsEnvSet("ADPILATUS","/corvette/home/epics/support/areaDetector-3-12-1/ADPilatus")
epicsEnvSet("ADPROSILICA","/corvette/home/epics/support/areaDetector-3-12-1/ADProsilica")
epicsEnvSet("ADPOINTGREY","/corvette/home/epics/support/areaDetector-3-12-1/ADPointGrey")
epicsEnvSet("ADEIGER","/corvette/home/epics/support/areaDetector-3-12-1/ADEiger")
epicsEnvSet("ADGENICAM","/corvette/home/epics/support/areaDetector-3-12-1/ADGenICam")
epicsEnvSet("ADSPINNAKER","/corvette/home/epics/support/areaDetector-3-12-1/ADSpinnaker")
epicsEnvSet("ADARAVIS","/corvette/home/epics/support/areaDetector-3-12-1/ADAravis")
epicsEnvSet("ADVIMBA","/corvette/home/epics/support/areaDetector-3-12-1/ADVimba")
epicsEnvSet("CARS","/corvette/home/epics/support/CARS")
epicsEnvSet("CRYOCONM32","/corvette/home/epics/support/CryoconM32")
epicsEnvSet("DAC128V","/corvette/home/epics/support/dac128V-2-10-1")
epicsEnvSet("DANTE","/corvette/home/epics/support/dante-1-1")
epicsEnvSet("DELAYGEN","/corvette/home/epics/support/delaygen-1-2-3")
epicsEnvSet("DXP","/corvette/home/epics/support/dxp-6-0")
epicsEnvSet("GALIL","/corvette/home/epics/support/Galil/3-6")
epicsEnvSet("IP","/corvette/home/epics/support/ip-2-21-1")
epicsEnvSet("IPAC","/corvette/home/epics/support/ipac-2-16")
epicsEnvSet("IP330","/corvette/home/epics/support/ip330-2-10")
epicsEnvSet("IPUNIDIG","/corvette/home/epics/support/ipUnidig-2-12")
epicsEnvSet("MCA","/corvette/home/epics/support/mca-7-9")
epicsEnvSet("MEASCOMP","/corvette/home/epics/support/measComp-4-1")
epicsEnvSet("MODBUS","/corvette/home/epics/support/modbus-3-2")
epicsEnvSet("MOTOR","/corvette/home/epics/support/motor-7-3")
epicsEnvSet("OPTICS","/corvette/home/epics/support/optics-2-13-5")
epicsEnvSet("PMAC","/corvette/home/epics/support/pmac-2-5-13")
epicsEnvSet("QUADEM","/corvette/home/epics/support/quadEM-9-4")
epicsEnvSet("SCALER","/corvette/home/epics/support/scaler-4-0")
epicsEnvSet("SOFTGLUE","/corvette/home/epics/support/softGlue-2-8-3")
epicsEnvSet("STD","/corvette/home/epics/support/std-3-6-3")
epicsEnvSet("STREAM","/corvette/home/epics/support/stream-2-8-24")
epicsEnvSet("SYMETRIE","/corvette/home/epics/support/motorSymetrie-1-0")
epicsEnvSet("SYRINGEPUMP","/corvette/home/epics/support/SyringePump-0-1")
epicsEnvSet("TOMOSCAN","/corvette/home/epics/support/tomoscan")
epicsEnvSet("VAC","/corvette/home/epics/support/vac-1-9-1")
epicsEnvSet("VME","/corvette/home/epics/support/vme-2-9-4")
dbLoadDatabase("../../dbd/CARSLinux.dbd")
CARSLinux_registerRecordDeviceDriver(pdbbase)
dbLoadRecords("string_test.db", "")
< ../save_restore_IOCSH.cmd
### save_restore setup
#
# The rest this file does not require modification for standard use, but...
# If you want save_restore to manage its own NFS mount, specify the name and
# IP address of the file server to which save files should be written.
# This currently is supported only on vxWorks.
#save_restoreSet_NFSHost("oxygen", "164.54.52.4")
# status-PV prefix
#save_restoreSet_status_prefix("xxx:")
# Debug-output level
save_restoreSet_Debug(0)
# Ok to save/restore save sets with missing values (no CA connection to PV)?
save_restoreSet_IncompleteSetsOk(1)
# Save dated backup files?
save_restoreSet_DatedBackupFiles(1)
# Number of sequenced backup files to write
save_restoreSet_NumSeqFiles(3)
# Time interval between sequenced backups
save_restoreSet_SeqPeriodInSeconds(300)
# specify where save files should be
set_savefile_path(".", "autosave")
###
# specify what save files should be restored.  Note these files must be
# in the directory specified in set_savefile_path(), or, if that function
# has not been called, from the directory current when iocInit is invoked
set_pass0_restoreFile("auto_positions.sav")
set_pass0_restoreFile("auto_settings.sav")
set_pass1_restoreFile("auto_settings.sav")
# load general-purpose interpolation tables with local, user-editable file
# (if interp_settings.req is included in auto_settings.req, the next line
# will overwrite those restored values)
#set_pass1_restoreFile("interp.sav")
###
# specify directories in which to to search for included request files
set_requestfile_path("./")
set_requestfile_path("/corvette/home/epics/support/CARS",     "db")
set_requestfile_path("/corvette/home/epics/support/CARS",     "iocBoot")
set_requestfile_path("/corvette/home/epics/support/areaDetector-3-12-1/ADCore",   "db")
set_requestfile_path("/corvette/home/epics/support/autosave-5-10-2", "db")
set_requestfile_path("/corvette/home/epics/support/busy-1-7-3",     "db")
set_requestfile_path("/corvette/home/epics/support/calc-3-7-4",     "db")
set_requestfile_path("/corvette/home/epics/support/dac128V-2-10-1",  "db")
set_requestfile_path("/corvette/home/epics/support/dxp-6-0",      "db")
set_requestfile_path("/corvette/home/epics/support/Galil/3-6",    "db")
set_requestfile_path("/corvette/home/epics/support/ip-2-21-1",       "db")
set_requestfile_path("/corvette/home/epics/support/mca-7-9",      "db")
set_requestfile_path("/corvette/home/epics/support/measComp-4-1", "db")
set_requestfile_path("/corvette/home/epics/support/motor-7-3",    "db")
set_requestfile_path("/corvette/home/epics/support/optics-2-13-5",   "db")
set_requestfile_path("/corvette/home/epics/support/quadEM-9-4",   "db")
set_requestfile_path("/corvette/home/epics/support/scaler-4-0",   "db")
set_requestfile_path("/corvette/home/epics/support/sscan-2-11-5",    "db")
set_requestfile_path("/corvette/home/epics/support/SyringePump-0-1", "db")
set_requestfile_path("/corvette/home/epics/support/std-3-6-3",      "db")
set_requestfile_path("/corvette/home/epics/support/tomoscan", "db")
set_requestfile_path("/corvette/home/epics/support/vme-2-9-4",      "db")
#dbLoadRecords("$(AUTOSAVE)/db/save_restoreStatus.db", "P=xxx:")
iocInit
Starting iocInit
############################################################################
## EPICS R7.0.7.1-DEV
## Rev. R7.0.7-125-g5a1f3ecc8b9d97a5fa4f-dirty
## Rev. Date Git: 2023-05-18 11:57:41 -0700
############################################################################
drvStreamInit: Warning! STREAM_PROTOCOL_PATH not set.
save_restore: Can't open file './autosave/auto_positions.sav'.
save_restore: Trying backup file './autosave/auto_positions.savB'
save_restore: Can't open file './autosave/auto_positions.savB'.
save_restore: Can't figure out which seq file is most recent,
save_restore: so I'm just going to start with './autosave/auto_positions.sav0'.
save_restore: Trying backup file './autosave/auto_positions.sav0'
save_restore: Can't open file './autosave/auto_positions.sav0'.
save_restore: Trying backup file './autosave/auto_positions.sav1'
save_restore: Can't open file './autosave/auto_positions.sav1'.
save_restore: Trying backup file './autosave/auto_positions.sav2'
save_restore: Can't open file './autosave/auto_positions.sav2'.
save_restore: Can't find a file to restore from...save_restore: ...last tried './autosave/auto_positions.sav2'. I give up.
save_restore: **********************************

save_restore: Can't open save file.cas WARNING: Configured TCP port was unavailable.
cas WARNING: Using dynamically assigned TCP port 41908,
cas WARNING: but now two or more servers share the same UDP port.
cas WARNING: Depending on your IP kernel this server may not be
cas WARNING: reachable with UDP unicast (a host's IP in EPICS_CA_ADDR_LIST)
iocRun: All initialization complete
2023-07-07T12:51:52.424 Using dynamically assigned TCP port 49353.
create_monitor_set("auto_settings.req", 30)
epics> auto_settings.sav: 2 of 2 PV's connected

So I don't get any autosave error, even though I have not passed the macro STAGE_TYPE when loading the database.

This is the resulting auto_settings.sav file:

corvette:CARS/iocBoot/ioc13test>more autosave/auto_settings.sav
# autosave R5.3 Automatically generated - DO NOT MODIFY - 230707-125223
string_test.VAL
string_test.DESC This is the description
<END>