vlead / feedback-portal

This is the portal, where all kinds of feedback related to a lab or an experiment is filled in.
Other
0 stars 11 forks source link

Feedback dump API timed out for huge number of feedback records #51

Open ksripathi opened 6 years ago

ksripathi commented 6 years ago

As one of the REST API of feedback-portal service is to take the dump of feedback records for a given date and store them into the elasticsearch database.

During the process of taking dump of all feedback records for a given date using (https://github.com/vlead/feedback-portal/blob/master/src/design/index.org#get-all-feedback-forms-for-a-date) due to huge number of records server is timed out hence It couldn't store the feedback records into elasticsearch database this issue leads to mis match of total feedback records in feedback-portal service and in elasticsearch database

for the following dates feedback records as follows DATE Records 31-03-2017 156 10-04-2017 122 11-04-2017 95 12-04-2017 540 Total 913 records Total records on feedback service as of on 18-April-2017 ----> 4073 Total feedback records on elasticsearch database as of 18-April-2017 ---> 3160 (4073-913)

Ideally records on both the services should be the same due this issue there is a mismatch

ksripathi commented 6 years ago

@sivashankerN

we should change the server configuration timed out value Can we prioritize this issue and work on it since this is the right time to finish up all the analytics related issues

ksripathi commented 6 years ago

URL : http://feedback.vlabs.ac.in/feedback_dump?date=31-03-2017&key=defaultkey

<!DOCTYPE

HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">

502 Proxy Error

Proxy Error

The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET /feedback_dump .

Reason: Error reading from remote server


Apache/2.2.15 (CentOS) Server at feedback.vlabs.ac.in Port 80
sivashankerN commented 6 years ago

temporary change : Timeout value is changed. You can try the feedback api for the results for permanent change, we need to modify systems model. let it be for now. If it compulsory, please et me know.

ksripathi commented 6 years ago

@sivashankerN Thank you