search

eomnl / sas-jobmon

EOM SAS Job Monitor
GNU General Public License v3.0
10 stars 7 forks source link

Unknown Error in 'Add Job Sequence Number' #12

Open richardpatersonGIT opened 6 years ago

richardpatersonGIT commented 6 years ago

I am receiving an error when executing the "proc display c=work.dimon.dimon_job_sequence.scl batch;" statement within the 'Add Job Sequence Number'.

It appears to be related to the number of input records. We currently have approx 100 flows. Running the ETL job with a subset of flows gives no error.

any ideas?

NOTE: PROC CIMPORT begins to create/update catalog WORK.DIMON NOTE: Entry DIMON_JOB_SEQUENCE.SCL has been imported. NOTE: Entry METHODS.SCL has been imported. NOTE: Total number of entries processed in catalog WORK.DIMON: 2

NOTE: PROCEDURE CIMPORT used (Total process time): real time 0.40 seconds user cpu time 0.03 seconds system cpu time 0.01 seconds memory 1468.96k OS Memory 15684.00k Timestamp 07/05/2018 10:40:18 AM Step Count 100 Switch Count 3

MPRINT(X): proc display c=work.dimon.dimon_job_sequence.scl batch; MPRINT(X): run;

2147484928 P  WORK à•Ÿ R_ ˆ  SASTMP-0000000 Program returning prematurely at line 179 AF Program: WORK.DIMON.DIMON_JOB_SEQUENCE.SCL 2147484928 P  WORK à•ŸR_ ˆ  SASTMP-0000000 NOTE: PROCEDURE DISPLAY used (Total process time): real time 28:38.00 user cpu time 27:56.94 system cpu time 7.34 seconds memory 5357631.10k OS Memory 5632532.00k Timestamp 07/05/2018 11:08:56 AM Step Count 101 Switch Count 0

ERROR: SAS ended due to errors. You specified: OPTIONS ERRORABEND;. ERROR: Errors printed on page 105.

bheinsius commented 6 years ago

I have not seen problems with this step before. Can you email me the complete log and - if possible - the data set that goes into the proc display?

richardpatersonGIT commented 6 years ago

here is the full log. The dataset will be an issue but I will try and get some metrics - number of flows, sub-flows, jobs etc.

Adhoc_Richard_DIMon_Load_Flows_and_Jobs_2018.07.05_10.30.05.log

richardpatersonGIT commented 6 years ago

Hi, I have attached a zip file containing 4 SAS datasets. These datasets have been created using the DIMON ETL job on the customer's ETL server. The 4 datasets are the inputs prior to starting the 'Add Job Sequence Number' transformation. To recap, the transformation ends after several hours processing with an error. It appears that the memory limit has been reached. Hope you can help with this issue. dimon_inputs.zip

bheinsius commented 6 years ago

I created a new branch https://github.com/bheinsius/eom-sas-dimon/tree/Add-Job-Sequence-Number-Fix and commited a fix to it. Please test.

richardpatersonGIT commented 6 years ago

Hi, I have tested the fix and this seems to have solved the issue. The ETL now runs to completion and I see all job flows, subflows and jobs. Many thanks for this!