Ingestion Job - Status/Progress is misleading

P3-Core-Dev-Team / P3-Q-A

This repo is for tracking features

Other

0 stars 0 forks source link

Ingestion Job - Status/Progress is misleading #5

Closed harsha-kotha closed 5 months ago

harsha-kotha commented 11 months ago

Environment - Pulte New Dev ADS 3.1.7

https://archon-datastore.platform3solutions.com/pulte/dev/login

App : Recruiting Management

Ingestion Status does not show actual status.

Table : CANDIDATE_BLOB Launch time 2023-07-23 14:21:30

Start time 2023-07-23 14:21:59

Current Time Approx : 2023-07-23 16:07:00

Message Ingestion in progress...

Almost 90 minutes has passed, Elapsed time show 00 ms and unable to know the current progress/status of the job. Elapsed time is very much misleading. There has to be better way to show the progress, current one is very misleading.

harsha-kotha commented 11 months ago

Pls tell what details needed, i have added as much information as possible. Selva has taken a look at it. Here is Selva's comment

_**Harsha I understand that the total number of files is 110,620, and each file size ranges from 10 KB to 70 KB only. If the file sizes are smaller and more, moving them to S3 will take time.

Also same ASSIGNMENT Table total size 10 GB (apprx) and total file 2300 file completed less then 1 hour.**_

For me the justification given is not acceptable.

kilam005 commented 11 months ago

After disucssion with Harsha the expectation is

Elapsed time should show the running time from launch time to current time if the job is in not in final state
If its final time the elaspsed time must show the value from launch time to end time if the job is in final state

⚠️Keep elapsed time a blank if launch time is unavaiable ℹ️Launch time will be available when the set is scheduled for ingestion process to start operation

Udhayanila8 commented 11 months ago

Issue has been fixed. Tested in EMR region.

ClementJosh21 commented 8 months ago

@harsha-kotha Fix was provided as per @kilam005 's comment above.

kilam005 commented 5 months ago

@harsha-kotha fixes are made as per the discussion we had.

fix details

After disucssion with Harsha the expectation is

Elapsed time should show the running time from launch time to current time if the job is in not in final state

If its final time the elaspsed time must show the value from launch time to end time if the job is in final state

⚠️Keep elapsed time a blank if launch time is unavaiable ℹ️Launch time will be available when the set is scheduled for ingestion process to start operation

kilam005 commented 5 months ago

@harsha-kotha : please check and provide your comments

harsha-kotha commented 5 months ago

Closing by Nila's retest comment.