Mothball unused maps and bring useful temps onto Ship-shape

[ColmDC]

Is your feature request related to a problem? Please describe. We are hosting and maintaining unused maps and some useful ones are not visible

Describe the solution you'd like Mothball the following maps, both prod and dev and their datafactory processes.

• [ ] https://prod.mutual-aid.solidarityeconomy.coop/
• [ ] https://dev.mutual-aid.solidarityeconomy.coop/
• [ ] https://dev.ica-youth-network.solidarityeconomy.coop/
• [ ] https://dev.ica-youth-network.solidarityeconomy.coop/
• [ ] https://dev.maps.solidarityeconomy.coop/temp/dotcoop2023/
• [ ] https://dev.maps.solidarityeconomy.coop/temp/merge/

Move the following ones off temp subdirectories and make visible on ShipShape

• [ ] https://dev.maps.solidarityeconomy.coop/temp/demo-merge-map/
• [ ] https://dev.maps.solidarityeconomy.coop/temp/demo-merge-map-plus/

[ColmDC]

Are there any other useful temp ones that might be worth making a little more visible? @wu-lee

[wu-lee]

The temp directory is for maps which are disposable or unreliably present or unreliably working.

The maps currently in the temp directory:

1. coopsuk
2. coopsuk-geoapify-new
3. coopsuk-geoapify-old
4. coopsuk-mapbox-new
5. demo-merge-map
6. demo-merge-map-plus
7. dotcoop2023
8. merge
9. obo-public
10. worker.coop

• 1 and 9 are builds of mykomap's bleeding-edge testing branch with a couple of representative cases. testing is ephemeral.
• 2, 3 and 4 have some use, but are still temporary and might not be worth listing shipshape
• 7 can be purged, maybe 8 too (it's got a bad name in any case)
• 10 is work in progress, and may be superceded with something non-temp when Graham gets back to us.

[ColmDC]

Added 7 and 8 to the mothball list above.

[ColmDC]

Noticed that https://dev.maps.solidarityeconomy.coop/icd/ is missing from ship-shape. When moving to dev-2, can we create a version there, on https://maps.coop/icd-23/.

[wu-lee]

Those mothballable maps on solidarityeconomy.coop domain probably just need to be omitted from the list when the others are moved across to dev-2 as part of the migration:

https://github.com/DigitalCommons/technology-and-infrastructure#workspaces/megaboard-639bb01fc526ef0010ab662b/epics/Z2lkOi8vcmFwdG9yL1plbmh1YkVwaWMvMTQzMTEy

I'd prefer to leave them alone this as reduces the things to do.

I will remove them from the datafactory (dev and prod-1) to reduce the work done there, or potentially. Well - of all of them, only mutual-aid is built by the data factory. (DONE)

And likewise I'll remove them from shipshape. (DONE)

What should I do with the mutual-aid spreadsheet, which is the source of the data? Leave it, delete it, or copy it and delete it?

The archived CSVs need a home - where's the best place for that?

[wu-lee]

And there's a new workers-coop map of course. I'll aim to ad that to shipshape too.

[ColmDC]

https://github.com/DigitalCommons/technology-and-infrastructure#workspaces/megaboard-639bb01fc526ef0010ab662b/epics/Z2lkOi8vcmFwdG9yL1plbmh1YkVwaWMvMTQzMTEy

Ah, we can see those Epics again!

[wu-lee]

The ICD map is now redeployed at https://dev.maps.coop/icd-23/

Shipshape is updated to point to that. (There's only a dev copy currently.) Note, there is no SPARQL endpoint because this map doesn't use SPARQL.

[wu-lee]

The demo-merge-map and demo-merge-map-plus maps are now redeployed at:

• https://dev.maps.coop/demo-merge-map
• https://dev.maps.coop/demo-merge-map-plus

And added to Shipshape. Again, only dev copies exist currently. Also, both these maps are built from different branches of the same map source code, and neither uses SPARQL or the datafactory, so fields will be missing. Data was committed to the source code for expediency. The data is generated in a rather ad-hoc fashion as well. Although it was scripted, and the scripts are committed to demo-merge-map, not all of the intermediate databases and data is. So it's basically just a proof of concept and still a pile of experiment - tech debt warning! (I'm currently wondering if the data deployed for these maps is the same as that deployed for the dev-1 copies, and which data is which [later: it's committed and to distinct CSV files, so maps should be ok]).

[wu-lee]

For safety I've made an interim commit of all the intermediate data (some of which are binary and probably don't belong in git but whatever).

https://github.com/DigitalCommons/demo-merge-map/commit/5b58728f4e779e4321d29f6465156fac5b3354fa

[wu-lee]

Ok, I think this is all done?

[ColmDC]

What should I do with the mutual-aid spreadsheet, which is the source of the data? Leave it, delete it, or copy it and delete it?

Where is it now? On our Nextcloud?

[ColmDC]

The archived CSVs need a home - where's the best place for that? How many are there? How much data? Do you think there is public value to them? What choices do we have. Nextcloud or Github. Not GoogleDrive?

[ColmDC]

Can't see dev.maps.coop/workers-coop on Ship-Shape?

Should ShipShape move to dev-2 too?

[wu-lee]

The amount of data in the CSVs isn't large. There'll be one original CSV for mutual-aid, and one final output CSV. In ICA-youth's cases, several originals, one output. I think ICAY originals might already have been in SEA NextCloud, so are presumably now in DCC Google Drive.

The other location we've used is this Github repository, which has deep-adaptation and ICA data (on different branches)

https://github.com/DigitalCommons/data-capture

I think there might be value to keeping them if we ever plan to re-start the maps.

[wu-lee]

Can't see dev.maps.coop/workers-coop on Ship-Shape?

Oops, missed that. It is now.

Should ShipShape move to dev-2 too?

Yes. I've deployed it there too, now. And updated the branding to DCC. There are now two copies, but I'll stop updating the dev-1 copy.

[ColmDC]

Did a quick run through the new moved Ship-Shape

[ColmDC]

I ringed all the broken links and maps that failed to load.

[ColmDC]

I think there might be value to keeping them if we ever plan to re-start the maps.

So are you suggesting https://github.com/DigitalCommons/data-capture ?

I can't see any problem in using it.

[ColmDC]

https://dev.mersey-green.solidarityeconomy.coop/ https://dev.maps.solidarityeconomy.coop/newbridge/ https://dev.oxford.solidarityeconomy.coop/ working now and have the correct link in ShipShape

[ColmDC]

The other ringed errors on Shipshape are local to shipshape I think.

[wu-lee]

When I check now:

• All the ringed maps in the "server" column seem to go to working maps.
• The broken links in the "dataset" column are I think a problem with ShipShape and newer Mykomaps which use CSV as a source instead of SPARQL.
• Of the "endpoint" links
  • all are on http URLs, so expect to see a warning about visiting an insecure site
  • dev-1's Virtuoso server seemed to have died, but works after being rebooted (probably killed by an exploding query)
• Those in the "DGU" column which go to 404s seem to be a lod.coop redirectory issue. Puzzling why some but not all fail though.

None of these are mothballing-related, I think. Need new issues for

• lod.coop redirectory
• fixing shipshape/mykomap links for CSV datasets

[ColmDC]

• All the ringed maps in the "server" column seem to go to working maps.

I confirm they all seem to work now.

[ColmDC]

• fixing shipshape/mykomap links for CSV datasets

https://github.com/DigitalCommons/shipshape/issues/4

[ColmDC]

• lod.coop redirectory

Added note in https://github.com/DigitalCommons/technology-and-infrastructure/issues/94