CCI-MOC / xdmod-cntr

A project to prototype the use of XDMOD with OpenStack and OpenShift on the MOC
1 stars 5 forks source link

xdmod ticket #33183 The script /usr/lib64/xdmod/batch_export_manager.php doesn't run to completion #226

Open rob-baron opened 1 year ago

rob-baron commented 1 year ago

Robert Bartlett Baron, reported about 2 months ago The data export section of xdmod is not currently working as it requires the script /usr/lib64/xdmod/batch_export_manager.php to run. Currently it is giving the following errors:

sh-4.2$ /usr/lib64/xdmod/batch_export_manager.php 2023-06-20 20:57:00 [notice] batch_export_manager start (process_start_time: 2023-06-20 20:57:00) 2023-06-20 20:57:01 [error] Unknown "batchExport" option 'true' (module: data-warehouse-export, stacktrace: #0 /usr/share/xdmod/classes/DataWarehouse/Data/BatchDataset.php(94): DataWarehouse\Data\RawStatisticsConfiguration->getBatchExportFieldDefinitions('Cloud')

1 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(309): DataWarehouse\Data\BatchDataset->__construct(Object(DataWarehouse\Query\Cloud\JobDataset), Object(XDUser), Object(CCR\Logger))

2 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(156): DataWarehouse\Export\BatchProcessor->getDataSet(Array, Object(XDUser))

3 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(118): DataWarehouse\Export\BatchProcessor->processSubmittedRequest(Array)

4 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(103): DataWarehouse\Export\BatchProcessor->processSubmittedRequests()

5 /usr/lib64/xdmod/batch_export_manager.php(89): DataWarehouse\Export\BatchProcessor->processRequests()

6 {main})

2023-06-20 20:57:01 [error] Failed to export data: Failed to create batch export query (module: data-warehouse-export, stacktrace: #0 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(156): DataWarehouse\Export\BatchProcessor->getDataSet(Array, Object(XDUser))

1 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(118): DataWarehouse\Export\BatchProcessor->processSubmittedRequest(Array)

2 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(103): DataWarehouse\Export\BatchProcessor->processSubmittedRequests()

3 /usr/lib64/xdmod/batch_export_manager.php(89): DataWarehouse\Export\BatchProcessor->processRequests()

4 {main})

postdrop: warning: unable to look up public/pickup: No such file or directory postdrop: warning: unable to look up public/pickup: No such file or directory 2023-06-20 20:57:01 [error] Unknown "batchExport" option 'true' (module: data-warehouse-export, stacktrace: #0 /usr/share/xdmod/classes/DataWarehouse/Data/BatchDataset.php(94): DataWarehouse\Data\RawStatisticsConfiguration->getBatchExportFieldDefinitions('Cloud')

1 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(309): DataWarehouse\Data\BatchDataset->__construct(Object(DataWarehouse\Query\Cloud\JobDataset), Object(XDUser), Object(CCR\Logger))

2 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(156): DataWarehouse\Export\BatchProcessor->getDataSet(Array, Object(XDUser))

3 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(118): DataWarehouse\Export\BatchProcessor->processSubmittedRequest(Array)

4 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(103): DataWarehouse\Export\BatchProcessor->processSubmittedRequests()

5 /usr/lib64/xdmod/batch_export_manager.php(89): DataWarehouse\Export\BatchProcessor->processRequests()

6 {main})

2023-06-20 20:57:01 [error] Failed to export data: Failed to create batch export query (module: data-warehouse-export, stacktrace: #0 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(156): DataWarehouse\Export\BatchProcessor->getDataSet(Array, Object(XDUser))

1 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(118): DataWarehouse\Export\BatchProcessor->processSubmittedRequest(Array)

2 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(103): DataWarehouse\Export\BatchProcessor->processSubmittedRequests()

3 /usr/lib64/xdmod/batch_export_manager.php(89): DataWarehouse\Export\BatchProcessor->processRequests()

4 {main})

postdrop: warning: unable to look up public/pickup: No such file or directory postdrop: warning: unable to look up public/pickup: No such file or directory postdrop: warning: unable to look up public/pickup: No such file or directory postdrop: warning: unable to look up public/pickup: No such file or directory 2023-06-20 20:57:09 [notice] batch_export_manager end (process_end_time: 2023-06-20 20:57:09)

Any suggestions? Robert Bartlett Baron , said about 2 months ago I got the script to run to completion, it actually wants a "public/pickup" directory from the directory that is is running in. For example:

sh-4.2$ /usr/lib64/xdmod/batch_export_manager.php 2023-06-27 21:24:11 [notice] batch_export_manager start (process_start_time: 2023-06-27 21:24:11) postdrop: warning: unable to look up public/pickup: No such file or directory 2023-06-27 21:24:14 [notice] batch_export_manager end (process_end_time: 2023-06-27 21:24:14) sh-4.2$ pwd / sh-4.2$ cd /root/xdmod_data sh-4.2$ /usr/lib64/xdmod/batch_export_manager.php 2023-06-27 21:24:38 [notice] batch_export_manager start (process_start_time: 2023-06-27 21:24:38) 2023-06-27 21:24:38 [notice] batch_export_manager end (process_end_time: 2023-06-27 21:24:38)

The '/' directory does not have a "public/pickup" directory from it, where as /root/xdmod_data does.

Running this does produce the requested export files.

I think we can close this for now. Ryan Rathsam , said about 2 months ago Ticket: https://help.xdmod.org/support/tickets/33183

Hi Robert,

Sounds good!

Thanks!

Ryan