Closed wseis closed 4 years ago
From my logs for the middlelayer I can see the following request several times
2020-03-09 13:37:50 [32minfo[39m [index.js]: request body
{
"spot_id": 43,
"user_id": 8
}
With the following error response
2020-03-09 13:37:55 [32minfo[39m [index.js]: response passed back from opencpu
{
"event": "response",
"payload": {
"data": {},
"success": [
false
],
"message": [
"Error in api_get_bathingspot(user_id) : length(data) == 1 is not TRUE\n"
]
},
"sessionID": "2db79bbc-1df2-4a46-9bc1-9abb025e6937"
}
@hsonne This seems to be an issue with fhpredcit?
SELECT "name",
"updatedAt",
"apiEndpoints",
"influenceRainwater",
"influenceAgriculture",
"influencePurificationPlant"
from bathingspot WHERE id = 43;
Key | Value |
---|---|
name | kleine Badewiese |
updatedAt | 2020-03-09 10:44:59.029918 |
apiEndpoints | {"dischargesUrl":"https://raw.githubusercontent.com/KWB-R/flusshygiene/gh-pages/q_tw.json"} |
influenceRainwater | yes |
influenceAgriculture | no |
influencePurificationPlant | yes |
SELECT * from bathingspot_prediction where "bathingspotId" = 43;
SELECT * from bathingspot_model WHERE "bathingspotId" = 43;
id | createdAt | version | updatedAt | rmodel | parameter | comment | evaluation | bathingspotId |
---|---|---|---|---|---|---|---|---|
9 | 2020-02-20 15:36:27.791669 | 2 | 2020-02-20 15:36:27.795098 | Deprecated. Binary model file has been uploaded. | conc_ec | {"formula":"log_e.coli ~ q_mean_abs_1 + r_mean_mean_12","N":0.0002,"BP":0.0696,"R2":0.4928,"n_obs":160,"stat_correct":false,"in50":5,"below90":5,"below95":5,"in95":5} | 43 | |
13 | 2020-03-09 10:44:56.431728 | 2 | 2020-03-09 10:44:56.43549 | Deprecated. Binary model file has been uploaded. | conc_ec | {"formula":"log_e.coli ~ q_mean_abs_1 + r_mean_mean_12","N":0.0005,"BP":0.0874,"R2":0.4903,"n_obs":160,"stat_correct":false,"in50":5,"below90":5,"below95":5,"in95":5} | 43 |
SELECT count(*) from rain WHERE "bathingspotId" = 43;
count |
---|
912 |
SELECT count(*) from bathingspot_measurement WHERE "bathingspotId" = 43;
count |
---|
248 |
SELECT count(*) from discharge where "bathingspotId" = 43;
count |
---|
4747 |
SELECT * from purification_plant WHERE "bathingspotId" = 43;
id | name | createdAt | updatedAt | bathingspotId | url |
---|---|---|---|---|---|
9 | Ruhleben | 2020-01-06 15:30:28.699472 | 2020-01-22 14:25:11.383426 | 43 | https://kwb-r.github.io/flusshygiene/ka_ruh.json |
SELECT count(*) from p_plant_measurement WHERE "purificationPlantId" = 9;
count |
---|
3471 |
I cannot reproduce this error in R. I improved the error message so that the (unexpected) structure of the returned object will be shown. Will be available in the next release. Currently on the dev branch:
https://github.com/KWB-R/fhpredict/commit/c6860bf14337914924b5c35f0cef1054578e8aae
Thank you. Can I install by sha? c6860bf14337914924b5c35f0cef1054578e8aae
e.g.
RUN R -e "remotes::install_github(\"kwb-r/fhpredict@c6860bf\", build_vignettes = FALSE, force = TRUE, upgrade = \"never\")"
Sorry, my fault! I found a bug in predict_quality()
. The user ID was not considered but always set to 3 (used in the workshop, I assume). I fixed the bug and got the promising result (see below).
Link to bug fix: https://github.com/KWB-R/fhpredict/commit/5847ab62fa1a4d731fd5acb324ecab609c96db36
@fabianmoronzirfas Yes, you can install by sha. Better: by branch name: @dev
> fhpredict::predict_quality(user_id = 8, spot_id = 43)
Reading all models from the database ... ok. (0.20s)
Reading model with id = 13 from the database ... ok. (0.07s)
Getting URLs to Radolan files between 2019-09-21 and 2019-09-30 ... ok. (2.22s)
Reading rain data from database ... ok. (0.55s)
Importing rain data block 1/1 ...
Reading and masking from raa01-sf_10000-1909211050-dwd---bin (1/10 = 10.0%)...
Downloading raa01-sf_10000-1909211050-dwd---bin with access key ... ok. (1.18s)
Reading and masking from raa01-sf_10000-1909221050-dwd---bin (2/10 = 20.0%)...
Downloading raa01-sf_10000-1909221050-dwd---bin with access key ... ok. (0.96s)
Reading and masking from raa01-sf_10000-1909231050-dwd---bin (3/10 = 30.0%)...
Downloading raa01-sf_10000-1909231050-dwd---bin with access key ... ok. (0.93s)
Reading and masking from raa01-sf_10000-1909241050-dwd---bin (4/10 = 40.0%)...
Downloading raa01-sf_10000-1909241050-dwd---bin with access key ... ok. (1.20s)
Reading and masking from raa01-sf_10000-1909251050-dwd---bin (5/10 = 50.0%)...
Downloading raa01-sf_10000-1909251050-dwd---bin with access key ... ok. (0.80s)
Reading and masking from raa01-sf_10000-1909261050-dwd---bin (6/10 = 60.0%)...
Downloading raa01-sf_10000-1909261050-dwd---bin with access key ... ok. (0.80s)
Reading and masking from raa01-sf_10000-1909271050-dwd---bin (7/10 = 70.0%)...
Downloading raa01-sf_10000-1909271050-dwd---bin with access key ... ok. (0.80s)
Reading and masking from raa01-sf_10000-1909281050-dwd---bin (8/10 = 80.0%)...
Downloading raa01-sf_10000-1909281050-dwd---bin with access key ... ok. (0.68s)
Reading and masking from raa01-sf_10000-1909291050-dwd---bin (9/10 = 90.0%)...
Downloading raa01-sf_10000-1909291050-dwd---bin with access key ... ok. (0.65s)
Reading and masking from raa01-sf_10000-1909301050-dwd---bin (10/10 = 100.0%)...
Downloading raa01-sf_10000-1909301050-dwd---bin with access key ... ok. (0.65s)
Inserting 10 rain data records into the database ... ok. (0.21s)
ok. (10.83s)
Reading rain data from database ... ok. (0.55s)
Reading rain data from database ... ok. (0.51s)
Reading discharge data from database ... ok. (2.27s)
Reading irradiances data from database ... ok. (0.09s)
Reading plant measurements data from database ... ok. (1.52s)
Removing element 'hygiene_spot43' from list of data frames ... ok. (0.00s)
Using 64 variables:
- 'r_radolan'
- 'r_radolan_abs_1'
- 'r_radolan_abs_2'
- 'r_radolan_abs_3'
- 'r_radolan_abs_4'
- 'r_radolan_abs_5'
- 'r_radolan_mean_12'
- 'r_radolan_mean_123'
- 'r_radolan_mean_1234'
- 'r_radolan_mean_12345'
- 'r_radolan_mean_2345'
- 'r_radolan_mean_345'
- 'r_radolan_mean_45'
- 'r_radolan_mean_234'
- 'r_radolan_mean_23'
- 'r_radolan_mean_34'
- 'r_mean'
- 'r_mean_abs_1'
- 'r_mean_abs_2'
- 'r_mean_abs_3'
- 'r_mean_abs_4'
- 'r_mean_abs_5'
- 'r_mean_mean_12'
- 'r_mean_mean_123'
- 'r_mean_mean_1234'
- 'r_mean_mean_12345'
- 'r_mean_mean_2345'
- 'r_mean_mean_345'
- 'r_mean_mean_45'
- 'r_mean_mean_234'
- 'r_mean_mean_23'
- 'r_mean_mean_34'
- 'q_1'
- 'q_1_abs_1'
- 'q_1_abs_2'
- 'q_1_abs_3'
- 'q_1_abs_4'
- 'q_1_abs_5'
- 'q_1_mean_12'
- 'q_1_mean_123'
- 'q_1_mean_1234'
- 'q_1_mean_12345'
- 'q_1_mean_2345'
- 'q_1_mean_345'
- 'q_1_mean_45'
- 'q_1_mean_234'
- 'q_1_mean_23'
- 'q_1_mean_34'
- 'q_mean'
- 'q_mean_abs_1'
- 'q_mean_abs_2'
- 'q_mean_abs_3'
- 'q_mean_abs_4'
- 'q_mean_abs_5'
- 'q_mean_mean_12'
- 'q_mean_mean_123'
- 'q_mean_mean_1234'
- 'q_mean_mean_12345'
- 'q_mean_mean_2345'
- 'q_mean_mean_345'
- 'q_mean_mean_45'
- 'q_mean_mean_234'
- 'q_mean_mean_23'
- 'q_mean_mean_34'
Reading predictions data from database ... ok. (0.06s)
No predictions data available for user_id = 8 and spot_id = 43.
Nothing to delete.
$data
[1] 449 450 451 452 453
$success
[1] TRUE
$message
[1] "5 predictions have been posted to the database."
@hsonne :) jiiiiihhaaaa. Can you just make a patch release with 0.10.1 with that fix ?
@hsonne I update the running setup to v0.10.1 but I still have problems with the predcition function.
I can confirm that the communication works. I added for testing purpose the sleep function again and can trigger it and get a response back. Also running a longer process like loading of the rain data works. But running the predict funtion gives me the following error.
"child process has died
In call:
tryCatch({
if (length(priority))
setpriority(priority)
if (length(rlimits))
set_rlimits(rlimits)
if (length(gid))
setgid(gid)
if (length(uid))
setuid(uid)
if (length(profile))
aa_change_profile(profile)
if (length(device))
options(device = device)
graphics.off()
options(menu.graphics = FALSE)
serialize(withVisible(eval(orig_expr, parent.frame())), NULL)
}, error = function(e) {
old_class <- attr(e, \"class\")
structure(e, class = c(old_class, \"eval_fork_error\"))
}, finally = substitute(graphics.off()))"
Can you review this. Also if possible it would be great to have some more logging to see what is going on and if the functions are still running. Currently it is a blackbox for me.
Hm, seems to be an opencpu issue, e.g. also reported here: https://github.com/opencpu/opencpu/issues/319
@hsonne So what does the https://cran.r-project.org/web/packages/RJDBC/index.html package do? Can we fix this?
@fabianmoronzirfas I do not know the package RJDBC. Here is another discussion about the same error. Does that help? https://github.com/opencpu/opencpu/issues/245
Hehe. Saw your mention pop up on the same issue while I was reading it :D
Can you try to raise the rlimit.nproc value inside /etc/opencpu/server.conf ?
From the manual https://opencpu.github.io/server-manual/opencpu-server.pdf
rlimit.nproc– Maximum number of concurrent processes that the server is allowed to use. Defaults to 50 processes
This is currently set to 100
Do we have that many processes?
I do not open any additional processes (as far as I am aware of). But I am calling the prediction function posterior_predict()
from the rstanarm package. Maybe this function opens further processes. Unfortunately, I cannot see this from the source code that can be found here:
https://github.com/stan-dev/rstanarm/blob/master/R/posterior_predict.R
meh. Build with v0.11.0 is failing
This issue is blocked by it is not possible to rebuild the image using different settings for testing. So this is on hold until this build error is solved
Hm. No change in kwb.flusshygiene since oct 2019 @hsonne do you have any idea why we can't build
Maybe this error is related to the rstan not to kwb.flusshygiene?
Would it be useful if we try to install packages from binary download instead of building them from source?
See https://github.com/stan-dev/rstan/wiki/Installing-RStan-on-Linux
Trying to increase the rlimit.nproc
in the running container just killed it. I also increased the memory for the container with no luck.
Yiiiiihhhhaaa. Thanks to our combined effort @hsonne 🙏 This error is fixed.
Für Badestelle 43, User 8 sollten eigentlich alle Daten vorhanden sein um aktuelle Vorhersagen zu machen. Druck auf Vorhersage für zur Meldung "your request is being processed", führt jedoch zu keinem (sichtbaren) Ergebnis.