Open marktani opened 6 years ago
@marktani this issue is also affecting shared-eu-west-1
any idea when this may be fixed? I know the export function is not high priority, but for us we're blocked from upgrading to Framework as we'd need to take a backup of our production data before upgrading.
Hey @mattfysh, after further investigation, this seems to be a problem when you're trying to export data "across regions". So if you're trying to export data from a project in shared-eu-west-1
, try to send the export request from a european IP.
This might help you work around this issue until it's fixed from our side 🙂
Hey @marktani - this worked! Using NordVPN, able to connect via UK and export. thanks ✌️
@marktani do you know if this will be fixed in the near term or would it be wise to configure a VPN?
alternatively, is there a way to clone an existing baas to another region. ie: clone an account from eu to us?
@marktani I just ran another data backup (can never be too careful!) and this time "Downloading lists..." failed, even after using NordVPN to connect via the UK.
Downloading nodes... 2460ms
Downloading lists... ✔
▸ The download of lists failed. You may get fragmented data. Request ID: eu-west-1:simple:cjekua381n6rg0143wm19a113
Downloading relations... 1030ms
Zipping export... ⣻
Zipping export... 116ms
Exported service to export-2018-03-10T03:57:29.967Z.zip
Do you know if this is okay to ignore? e.g. would I still be able to restore (import) this data to production or another GC target?
my question just this : @marktani how to trying this "send the export request from a european IP"..??
this is just info about my log :
My problems exactly same, when try to export nodes from shared-eu-west-1
..
i already try using raw or using graphcool export -t dev.. the zip already downloaded but,
in log :
Downloading lists... 1313ms Downloading relations... ✔ ▸ The download of relations failed. You may get fragmented data. Request ID: ap-northeast-1:simple:cjekt7sw1xe5p0127wmnqyw2c Zipping export... Zipping export... 45ms Exported service to export-2018-03-10T03:27:40.669Z.zip You can import it to a new service with $ graphcool-framework import --source export-2018-03-10T03:27:40.669Z.zip --target target-name
when i open zip folder, there is no info about my nodes etc..
Rather than use a VPN I've managed to export my data using Google Cloud
create an F1-micro compute engine instance in Google Cloud with NodeJS (https://console.cloud.google.com/launcher/details/click-to-deploy-images/nodejs) in the region your GraphCool project is in (ie: eu-west).
SSH into the machine with the google web based terminal
run npm install graphcool-framework -g
run sudo nano .graphcoolrc
paste in your .graphcoolrc
file from your project
run (replacing your token and target)
GRAPHCOOL_PLATFORM_TOKEN=__YOUR_PLATFORM_TOKEN__ gcf export -e backup.zip -t __TARGET_FROM_GRAPHCOOLRC__
wait
If you don't know what folder you're in, run pwd
it should be something like /home/username
Choose Download file
from the top-right context menu
Type /home/username/backup.zip
(using your username and filename you set in step 6
...
profit?
I did essentially the same thing with AWS EC2 instance in the same region as GraphCool.
@marktani any thoughts on why I'm getting "The download of lists failed" but nodes and relations are okay? thx!
I'm experiencing this issue from a UK based IP.
Downloading nodes... 980ms
Downloading lists... ✔
▸ The download of lists failed. You may get fragmented data. Request ID:
▸ eu-west-1:simple:cjgpl4gvepcqz01714kcycdd9
Downloading relations... 238ms
Zipping export...
Zipping export... 35ms
Exported service to export-2018-05-02T20:59:28.864Z.zip
You can import it to a new service with
$ graphcool import --source export-2018-05-02T20:59:28.864Z.zip --target target-name
Please note this affects the "import" command too.
After dumping the fetch() call for the import the error response { requestId: 'ap-northeast-1:simple:XXXX', }
points to the wrong cluster. This should be shared-eu-west-1, not ap-northeast-1
@marktani Any updates on this? The export is holding some of my projects :(
Use one of the work-arounds. Either VPN or EU AWS EC2 instance.
@baerrach tried VPN from Europe, no luck.
It worked for me.
I can only assume your cluster is not in the EU region.
Your .graphcoolrc file will list your targets and the cluster it belongs too.
Make sure you connect from an IP that the CDN will recognise as one close to that region.
It could also be related with #544 instead, in which case you're out of luck since I doubt it'll be fixed
@baerrach I have tried to deploy it in the 3 regions: shared-eu-west-1
, shared-us-west-2
and shared-ap-northeast-1
, VPN from Europe, same issue. Also, whenever I try to export, I noticed that the region is always us-west-2
which is my initial region.
@hakimelek How are you noticing what the region is when exporting? When I run my export command with DEBUG=* there is no information about regions at all.
@baerrach I can see that in the requestId
WARNING: Error: The download of relations failed. You may get fragmented data. Request ID: us-west-2:simple:xxxxxxxxxxxxxxxxxxxx
Digging more into the data returned from the request, this is the data, that is causing the error.
{ requestId: 'us-west-2:simple:xxxxxxxxxxxxxxxxxxxx',
error: 'Whoops. Looks like an internal server error. Please contact us from the Console (https://console.graph.cool) or via email (support@graph.cool) and include your Request ID: us-west-2:simple:xxxxxxxxxxxxxxxxxxxx' }
@hakimelek Any updates on this case? I am having the same errors. Tried to run an instance on EC2 (eu-west-1) but the problem still happens.
@danielnakasato I had to resort to writing a manual script that exports the data in NDF (https://www.graph.cool/docs/reference/data-import-and-export/normalized-data-format-teroo5uxih)
@mattfysh Thanks for the answer! Just one more question... Did you manage to keep the objects IDs? I have some external services whose identifiers were created considering the user IDs...
Good question! The json files you create in NDF format will include existing system metadata including ID's, created/updated timestamps. As long as you import using the CLI, the target will retain the object IDs.
@danielnakasato I ended up just making different queries to all my database and download the data as json files then built a small script to make the migration, not the best but worked for me :/
Save yourself from messing around with EC2 or VPN.
Just query a nameserver in the region your looking to export from... For EU you can do something like:
dig @78.35.40.149 api.graph.cool
...
;; ANSWER SECTION:
api.graph.cool. 60 IN A 52.222.161.23
api.graph.cool. 60 IN A 52.222.161.75
...
Then just put one of the IPs into your /etc/hosts ...
52.222.161.130 api.graph.cool
Retry your export and it will work :)
@sreuter I've done as you've suggested, but it doesn't work for me...I get the exact same error. When I ping api.graph.cool after updating /etc/hosts, I get the result from the correct IP address. I'm not sure what's wrong.
@lastmjs Which region is your cluster and which ip did you put into the hosts file?
Nevermind, I ended up spinning up an EC2 instance in the same region and that didn't even work. Eu-west-1 I believe. I tried all of the IP addresses returned from the dig command
They were essentially the same as yours, 52.222.161.x
I've faced the same problem. It only worked for me when I change the region on my compute engine instance in Google Cloud.
CE - us-west1-a
Graphcool (shared cluster hosted) - shared-eu-west-1
Thank you @tsdexter you help me a lot. :pray:
Hello,
I'm trying to export data from an asian cluster but it's not working on any of them. However it's working on every eu server.
To reproduce, I just use the same schema and deployed it on eu and asian server.
Maybe related to #1565