search

aodn / content

Tracks AODN Portal content and configuration issues
0 stars 0 forks source link

Automatic upload of data from matlab affected by change to FTP with TLS #488

Closed DelphiWard closed 2 years ago

DelphiWard commented 2 years ago

@bpasquer @lbesnard @leonardolaiolo @ggalibert @ana-berger @mhidas

Any data that is being automatically transferred from the facilities from matlab script are likely to have been blocked by the recent switch to FTP TLS.

Matlab doesn't support ftp tls, so facilities that use that workflow will need to change the way they provide data. Options could be to write a shell script to it, or switch to python which does support ftp tls.

lbesnard commented 2 years ago

ptions could be to write a shell script to it, or switch to python which does support ftp tls.

I don't think this is our issue. FTP/RSYNC are quite basic tools data providers have to work with.

bpasquer commented 2 years ago

I agree, but their process might be silently failing. I realized today that the ANFOG realtime data workflow stopped when switched to FTP over TLS. @DelphiWard message is about making us aware that data flow from facilities that use automated Matlab script to transfer data via FTP may have got interrupted since the change.

lbesnard commented 2 years ago

letting them know is fine of course, but what I meant is that i don't think we should have to write them custom bash scripts

bpasquer commented 2 years ago

letting them know is fine of course, but what I meant is that i don't think we should have to write them custom bash scripts

Of course not, but we can provide guidance which is what @DelphiWard is saying.

ggalibert commented 2 years ago

@bpasquer can you please confirm which facilities were impacted by this problem (Matlab and FTP)? And if they all are now fixed? If so we can close this issue.

bpasquer commented 2 years ago

SOOP-BA, SOOP-CO2 and ANFOG were affected. SOOP_CO2 and SOOP_BA have resolved the issue. ANFOG will move away from FTP and implement rsync I will close once rsync is setup and data from ANFOG RT is flowing

ggalibert commented 2 years ago

@bpasquer looks like ANFOG RT data is flowing again so I assume they can now upload their data with rsync? Can you please confirm and close?

bpasquer commented 2 years ago

the upload of data from the glider facility has resumed. The upload is now done using rsync