search

Helioviewer-Project / jp2gen

IDL based tools to convert FITS data into JPEG2000 files for use with Helioviewer Project clients.
2 stars 5 forks source link

SDO/AIA data gaps Nov and Dec 2016 #27

Closed takucera closed 4 months ago

takucera commented 6 years ago

Hi, I am using JHelioviewer to access AIA data via IAS. I am looking at November and December 2016. There a number of multi-day gaps in the data. The data do seem to exist - I can see images from those times on the SDO pages. The gaps are (roughly) 2016/11/05 00:00 - 2016/11/08 00:00 2016/11/13 03:46 - 2016/11/16 00:00 2016/12/04 00:00 - 2016/12/06 00:00 2016/12/16 00:00 - 2016/12/21 00:00

I am guessing if there are this many gaps in these two months there are others as well. Terry

wafels commented 6 years ago

Terry,

There are definitely some files missing here at GSFC spanning the third time range you sent. I am checking all days in the range 2016/11/05 - 2016/12/21 just to make sure. These data should be downloaded to the server at GSFC in the next couple of days. I will let IAS know when the updating processing is complete.

wafels commented 6 years ago

over 38,000 files missing, or about a couple of days worth of what we normally expect from the AIA/HMI processing stream.

bogdanni commented 4 years ago

Are those data gaps now filled?

takucera commented 4 years ago

Hi, It is better, but there still seems to be some missing data. I am looking at 171 A and seeing a gap from about May 31 6:00 UT – June 1, 00:00 UT.

Is there no way to run some routine over the data base to search for gaps? Terry

From: Bogdan Nicula notifications@github.com Reply-To: Helioviewer-Project/JHelioviewer-SWHV reply@reply.github.com Date: Wednesday, November 13, 2019 at 5:31 AM To: Helioviewer-Project/JHelioviewer-SWHV JHelioviewer-SWHV@noreply.github.com Cc: "Kucera, Therese A. (GSFC-6710)" therese.a.kucera@nasa.gov, Author author@noreply.github.com Subject: [EXTERNAL] Re: [Helioviewer-Project/JHelioviewer-SWHV] SDO/AIA data gaps Nov and Dec 2016 (#65)

Are those data gaps now filled?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Helioviewer-2DProject_JHelioviewer-2DSWHV_issues_65-3Femail-5Fsource-3Dnotifications-26email-5Ftoken-3DAHQIINZH65QGY3WG632P773QTPJPHA5CNFSM4EZK6U52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOED5VERA-23issuecomment-2D553341508&d=DwMCaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=pFlbkPz2eKJikl2YwSWdRAxuGvJPmy8C0vB0BKuRQZA&m=2a9Mo17z1eV8tRu5bzRw4bvdUKxVZvBg0aTaaglrwLA&s=i_QFoZUKS3VoKE1S8Ay1rCcELbdz5Pg49CY1hK7hNyU&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AHQIINZI2ALABZ2Y7F5SNNDQTPJPHANCNFSM4EZK6U5Q&d=DwMCaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=pFlbkPz2eKJikl2YwSWdRAxuGvJPmy8C0vB0BKuRQZA&m=2a9Mo17z1eV8tRu5bzRw4bvdUKxVZvBg0aTaaglrwLA&s=hbSN5LiXOcrrggO0U98AnWfO2dQUFJGRDs3G_JOW8dw&e=.

ebuchlin commented 4 years ago

Is there no way to run some routine over the data base to search for gaps?

If such a routine exists we are interested in knowing more about it!

wafels commented 4 years ago

There is not a routine to do this.

The image timeline functionality implements a count of the number of images for different bin sizes. I am not entirely sure that that functionality works as expected.

We did implement some functionality to track image download stats, but it has been a long time since we looked at it. Let me look at this again and see how we can improve.

On Wed, Nov 13, 2019 at 1:21 PM ebuchlin notifications@github.com wrote:

Is there no way to run some routine over the data base to search for gaps?

If such a routine exists we are interested in knowing more about it!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Helioviewer-Project/JHelioviewer-SWHV/issues/65?email_source=notifications&email_token=AAHQEF4V3W4XPT6O3XL73BDQTRASTA5CNFSM4EZK6U52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOED7EOXI#issuecomment-553535325, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHQEF5HEV6BATSQNSZVJ3DQTRASTANCNFSM4EZK6U5Q .

wafels commented 4 months ago

According to the coverage plots (https://api.helioviewer.org/statistics/bokeh/coverages/SDO_coverage.html) these images are on the server. I'm closing this issue as completed.