search

bcgov / climr

An R package for downscaling monthly climate data for North America
https://bcgov.github.io/climr/
16 stars 5 forks source link

gcm data is limited to PNW #213

Closed cmahony closed 8 months ago

cmahony commented 8 months ago

library(rworldmap) data("countriesCoarse")

par(mar=c(1,1,1,1), mfrow=c(3,5)) for(i in 1:length(list_gcm())){ dbCon <- data_connect() gcm <- gcm_input( dbCon = dbCon, bbox = c(80, 20, -50, -170), gcm = list_gcm()[i], ssp = list_ssp()[2], period = list_gcm_period()[2], )

plot(countriesCoarse, xlim=c(-170, -50), ylim=c(20,80), main=list_gcm()[i]) plot(gcm[[1]][[1]], add=T, legend=F) }

Rplot

kdaust commented 8 months ago

image Fixed in https://github.com/bcgov/climr/commit/5c3968baf120b71c42123c244b0e8e208e288acf

cmahony commented 8 months ago

@kdaust i tried the reprex but it crashed midway and appears to have broken the server connection. can you try?

image

Not fully cached :( Will download more
Downloading GCM anomalies
.Caching data...
Downloading GCM anomalies
.Caching data...
Downloading GCM anomalies
.Caching data...
Not fully cached :( Will download more
Downloading GCM anomalies
.Caching data...
Not fully cached :( Will download more
Downloading GCM anomalies
.Caching data...
Not fully cached :( Will download more
Downloading GCM anomalies
.Caching data...
Not fully cached :( Will download more
Downloading GCM anomalies
.Caching data...
Downloading GCM anomalies
.Caching data...
Downloading GCM anomalies
.Caching data...
Not fully cached :( Will download more
Downloading GCM anomalies
.Caching data...
Not fully cached :( Will download more
Downloading GCM anomalies
.WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and repeat your command.

Error: Failed to fetch row : SSL SYSCALL error: EOF detected
WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and repeat your command.

<pool> Failed to activate and/or validate existing object.
<pool> Trying again with a new object.
Error: connection to server at "146.190.244.244", port 5432 failed: Connection refused (0x0000274D/10061)
    Is the server running on that host and accepting TCP/IP connections?
later: exception occurred while executing callback: 
invoke_wrapped: throwing std::runtime_error
WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and repeat your command.

<pool> Failed to activate and/or validate existing object.
<pool> Trying again with a new object.
Error: connection to server at "146.190.244.244", port 5432 failed: Connection refused (0x0000274D/10061)
    Is the server running on that host and accepting TCP/IP connections?
later: exception occurred while executing callback: 
invoke_wrapped: throwing std::runtime_error
WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and repeat your command.

<pool> Failed to activate and/or validate existing object.
<pool> Trying again with a new object.
Error: connection to server at "146.190.244.244", port 5432 failed: Connection refused (0x0000274D/10061)
    Is the server running on that host and accepting TCP/IP connections?
later: exception occurred while executing callback: 
invoke_wrapped: throwing std::runtime_error
WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and repeat your command.

<pool> Failed to activate and/or validate existing object.
<pool> Trying again with a new object.
Error: connection to server at "146.190.244.244", port 5432 failed: Connection refused (0x0000274D/10061)
    Is the server running on that host and accepting TCP/IP connections?
later: exception occurred while executing callback: 
invoke_wrapped: throwing std::runtime_error
<pool> Failed to activate and/or validate existing object.
<pool> Trying again with a new object.
Error: connection to server at "146.190.244.244", port 5432 failed: Connection refused (0x0000274D/10061)
    Is the server running on that host and accepting TCP/IP connections?
later: exception occurred while executing callback: 
invoke_wrapped: throwing std::runtime_error
WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and repeat your command.

<pool> Failed to activate and/or validate existing object.
<pool> Trying again with a new object.
Error: connection to server at "146.190.244.244", port 5432 failed: Connection refused (0x0000274D/10061)
    Is the server running on that host and accepting TCP/IP connections?
later: exception occurred while executing callback: 
invoke_wrapped: throwing std::runtime_error
WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and repeat your command.

<pool> Failed to activate and/or validate existing object.
<pool> Trying again with a new object.
Error: connection to server at "146.190.244.244", port 5432 failed: Connection refused (0x0000274D/10061)
    Is the server running on that host and accepting TCP/IP connections?
later: exception occurred while executing callback: 
invoke_wrapped: throwing std::runtime_error
WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and repeat your command.

<pool> Failed to activate and/or validate existing object.
<pool> Trying again with a new object.
Error: connection to server at "146.190.244.244", port 5432 failed: Connection refused (0x0000274D/10061)
    Is the server running on that host and accepting TCP/IP connections?
later: exception occurred while executing callback: 
invoke_wrapped: throwing std::runtime_error
> library(rworldmap)
WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and repeat your command.

<pool> Failed to activate and/or validate existing object.
<pool> Trying again with a new object.
Error: connection to server at "146.190.244.244", port 5432 failed: Connection refused (0x0000274D/10061)
    Is the server running on that host and accepting TCP/IP connections?
later: exception occurred while executing callback: 
invoke_wrapped: throwing std::runtime_error
> data("countriesCoarse")
> par(mar=c(1,1,1,1), mfrow=c(3,5))
> for(i in 1:length(list_gcm())){
+   dbCon <- data_connect()
+   gcm <- gcm_input(
+     dbCon = dbCon,
+     bbox = c(80, 20, -50, -170),
+     gcm = list_gcm()[i],
+     ssp = list_ssp()[2],
+     period = list_gcm_period()[2],
+   )
WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and repeat your command.

<pool> Failed to activate and/or validate existing object.
<pool> Trying again with a new object.
Error: connection to server at "146.190.244.244", port 5432 failed: Connection refused (0x0000274D/10061)
    Is the server running on that host and accepting TCP/IP connections?
later: exception occurred while executing callback: 
invoke_wrapped: throwing std::runtime_error
<pool> Failed to activate and/or validate existing object.
<pool> Trying again with a new object.
Error: connection to server at "146.190.244.244", port 5432 failed: Connection refused (0x0000274D/10061)
    Is the server running on that host and accepting TCP/IP connections?
later: exception occurred while executing callback: 
invoke_wrapped: throwing std::runtime_error
Restarting R session...

> library(climr)
Error in .External(list(name = "CppMethod__invoke_notvoid", address = <pointer: (nil)>,  : 
  NULL value passed as symbol address
Downloading and Caching ESM run info :)
Error in (function (classes, fdef, mtable)  : 
  unable to find an inherited method for function ‘poolClose’ for signature ‘"NULL"’
In addition: Warning messages:
1: In value[[3L]](cond) :
  Could not connect to database. Will try using cached data.
2: In fun(libname, pkgname) :
  Could not connect to server. Only cached normal periods will be available.
cmahony commented 8 months ago

i got it to run again but it crashed in the same iteration.

kdaust commented 8 months ago

I'll take a look shortly

kdaust commented 8 months ago

@cmahony Ok so the reason the server was crashing was due to how you were pulling the data - you were opening new database connections inside the loop and never closing them, so after a while it overwhelmed the server. I did find an issue with MIROC6 though, which is now fixed.

Reprex: 

library(rworldmap)
library(climr)
library(pool)
data("countriesCoarse")

par(mar=c(1,1,1,1), mfrow=c(3,5))
dbCon <- data_connect()
gcm <- gcm_input(
  dbCon = dbCon,
  bbox = c(80, 20, -50, -170),
  gcm = list_gcm(),
  ssp = list_ssp()[2],
  period = list_gcm_period()[2],
)

for(i in 1:length(list_gcm())){
  plot(gcm[[i]][[7]], legend=F, main=list_gcm()[i])
  plot(countriesCoarse, add = T, xlim=c(-170, -50), ylim=c(20,80))
}
poolClose(dbCon)

Output: GCM_Periods