Sunbird-cQube / community

Community repo for cQube where documentation, discussions, etc are enabled.
MIT License
3 stars 7 forks source link

Azur - Diksha_data_summary file processing is not happening #656

Closed chetandg123 closed 1 year ago

chetandg123 commented 2 years ago

Emitted the diksha tpd enrolment file to diksha_enrolment folder , file is not processing in static data processor

krishnakr6 commented 2 years ago

Azure support is not implemented in the below code. https://github.com/Sunbird-cQube/cQube_Edu/blob/cqube-v4.0-beta/processing/datasource/python/diksha_enrolment_file_split.py

@Shoaib15khan add the azure support here.

chetandg123 commented 2 years ago

@htvenkatesh @padmanabha1234 @Shoaib15khan emitted the diksha_data_summary_UP_76dis.zip file to Diksha folder , nifi processing is not happening

htvenkatesh commented 2 years ago

@padmanabha1234 @Shoaib15khan wts the update on this ticket?

devu1454 commented 2 years ago

@htvenkatesh re-tested this issue, still nifi is not picking the file from emission data storage

devu1454 commented 2 years ago

Re-tested this issue, Still nifi is not picking the diksha_data_summary adn diksha_tpd file

murali003 commented 2 years ago

@devu1454 you have checked older version can you please take latest updated code and re-test.

devu1454 commented 2 years ago

@murali003 Installed the updated code and checked the data processing of diksha files. Still i am unable to process the diksha_data_summary.zip file

murali003 commented 2 years ago

TDD:

  1. Emit the diksha_data_summary enrolment file to diksha_enrolment folder azure: diksha_data_summary file should be pick from emission bucket
  2. once it's pick the file all validation should be happen

Expected Result:

  1. Data should be updated in database table
  2. In log_summary folder, log_summary_diksha.json file to be generated
RNarmatha commented 2 years ago

@chetandg123  Please do the retest.

chetandg123 commented 2 years ago

Task is paused due to dependency of issue #688

chetandg123 commented 2 years ago

@padmanabha1234 diksha tpd enrolment file processing is not working

Image

murali003 commented 2 years ago

@Shoaib15khan the issue is related to diksha enrolment file, so please look into it.

Shoaib15khan commented 2 years ago

@chetandg123 issue is resolved . please take a retest on it. @RNarmatha FYI.

chetandg123 commented 2 years ago

@htvenkatesh Diksha all files are processed and we are able to see the ui report with data