gcp-pdp / geo-openstreetmap

32 stars 6 forks source link

BQ data does not update #38

Open GiancarloPera opened 2 years ago

GiancarloPera commented 2 years ago

It seems like updates are no more running and tables on Big Query public dataset are stopped at the version of 15 Nov 2021.

Is it an issue of the flow or there will be no more updates???

andrewscarani commented 2 years ago

Is there any maintainer of this project anymore?

andrewscarani commented 2 years ago

@allenday Is this project and/or is the BigQuery public dataset for OSM data still maintained at all by Google?

andrewscarani commented 1 year ago

@allenday @Pseverin @obsh @Jesus89

It seems that this data has not been updated in more than a year now... Does Google have any plans to continue regularly updating a hosted public BigQuery OSM dataset?

Screen Shot 2022-11-09 at 4 09 08 PM Screen Shot 2022-11-09 at 4 09 47 PM
ghost commented 1 year ago

andrewscarani did you ever get any news? Is this officialy dead since Nov 8th 2021? The tables display having been updated in September 2022, but the osm_timestamps go back to 2021..

andrewscarani commented 1 year ago

@Romeo-PHILLIPS No news. In fact, another employee recently notified us that some of BigQuery's weather data (outside of OSM) has not been updated in some time either. I may try to reach out through our Google reps and see if there's any way to get this fixed.

allenday commented 1 year ago

Hi everyone,

These tables are no longer being maintained. We were already short manpower from ~Nov 2021, and now Cloud infrastructure budget from EOY 2022.

I'll resurrect if and when I can. It could be a long while.

-Allen

andrewscarani commented 1 year ago

Hey @allenday, thank you for the response.

I'm just curious, if someone were to host this pipeline as-is in their own Google project, do you think this would provide sufficient data, or are there other fundamental issues existing in this pipeline that would make this insufficient?

From a technical or resource perspective, how much ongoing maintenance is required for such a pipeline?

allenday commented 1 year ago

It's certainly possible and I would be more than happy to help. I may also be able to share cost estimates, I believe one of the projects we used during development is dormant and still has historical billing data available.

On Wed, Feb 1, 2023 at 00:36 andrewscarani @.***> wrote:

Hey @allenday https://github.com/allenday, thank you for the response.

I'm just curious, if someone were to host this pipeline as-is in their own Google project, do you think this would provide sufficient data, or are there other fundamental issues existing in this pipeline that would make this insufficient?

From a technical or resource perspective, how much ongoing maintenance is required for such a pipeline?

— Reply to this email directly, view it on GitHub https://github.com/gcp-pdp/geo-openstreetmap/issues/38#issuecomment-1410706137, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADKOPZKSP7J7HCRCBW6XIDWVE5SNANCNFSM5YCLE2FQ . You are receiving this because you were mentioned.Message ID: @.***>

-- Schedule a call: https://meet.allen.day/call

ghost commented 1 year ago

It's certainly possible and I would be more than happy to help. I may also be able to share cost estimates, I believe one of the projects we used during development is dormant and still has historical billing data available. On Wed, Feb 1, 2023 at 00:36 andrewscarani @.> wrote: Hey @allenday https://github.com/allenday, thank you for the response. I'm just curious, if someone were to host this pipeline as-is in their own Google project, do you think this would provide sufficient data, or are there other fundamental issues existing in this pipeline that would make this insufficient? From a technical or resource perspective, how much ongoing maintenance is required for such a pipeline? — Reply to this email directly, view it on GitHub <#38 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADKOPZKSP7J7HCRCBW6XIDWVE5SNANCNFSM5YCLE2FQ . You are receiving this because you were mentioned.Message ID: @.> -- Schedule a call: https://meet.allen.day/call

If one of the issues is also cost, I would also be willing to help aswell. It was incredebily useful to have a direct way to interact with OSM data

AbhiramiVS commented 1 year ago

Hi, any update on this issue? Do we have a BQ table with updated OSM data?