search

DigitalCommons / mykomap

A web application for mapping initiatives in the Solidarity Economy
3 stars 0 forks source link

Publish working draft of Workers.coop MykoMap #207

Closed ColmDC closed 5 months ago

ColmDC commented 1 year ago

Description

[Workers.coop] Using the data we get from worker.coop CiviCRM stream, publish a MykoMap. Configure the map to use the following styles

Font Arial or Rubik for Normal text body. Titles, headings and stand-out text in Rubik.

Text Size Black or Extra bold for titles, and Rubik 11 or 12 for Normal text body. For titles, depending on document type 40 or 18.

Colours Our colour is Pantone 2012 which breaks down as rgb (239,150,0) , #ef9600 and black, with Pantone used for headings, subheadings, and other stand out features.

Headings Headings and titles can be in Bold Pantone 2012.

Acceptance Criteria

wu-lee commented 8 months ago

Ok, have munged the data - a preliminary version, some fields still to be updated (mainly labels -> IDs)

https://dev.data.digitalcommons.coop/workers-coop

And a map is deployed, although again preliminary - the popup hasn't been customised, etc.

https://dev.maps.coop/workers-coop/

ColmDC commented 8 months ago

Nice

ColmDC commented 8 months ago

csv data looks clean. Other than the url trimming, did you need to do any cleaning ot our end?

ColmDC commented 8 months ago

What's still to do to complete the map, (seperately from the change request side?).

wu-lee commented 8 months ago

No trimming beyond the URIs.

Outstanding:

ColmDC commented 8 months ago

  • I'm not sure if we have agreed the content of the pop-ups but if not that needs doing

  • ditto about page

  • ditto filterable fields

  • ditto searchable fields

See https://docs.google.com/document/d/1WkHl4oGmb3dtkcsdUqlGBx_mq9nWP1AoJNLd0-9HPc8/edit?usp=sharing

ColmDC commented 8 months ago

  • Data feed needs to supply IDs instead of labels for taxonomy fields

  • Corresponding indexes for IDs to labels need to be supplied and converted into vocab indexes

  • We probably need to know if we should filter out any rows by (say) status - currently Registered Status of "Dead" is omitted, historical reasons, nothing else.

These are all in Graham's court? Does he know exactly what he needs to do?

wu-lee commented 8 months ago

First two points yes, explicitly discussed at last meeting. Last one is on us, we just need to ask what to filter.

wu-lee commented 8 months ago

I've now gotten the dev.lod.coop redirector to work for worker-coop URLs. Required a small hack and an update of the Apache deployments to support that, in that the assumption to date was that all the lod.coop URLs being redirected had the same target base URL, whereas now we're transitioning from (dev.)data.solidarityeconomy.coop to (dev.)data.digitalcommons.coop that's not true. This base URL was set in the Apache config.

(While I was there I added support for ncba URLs; dotcoop2023 URLs were the other ones missing but they're now no longer used)

ColmDC commented 8 months ago

Required a small hack and an update of the Apache deployments to support that, in that the assumption to date was that all the lod.coop URLs being redirected had the same target base URL, whereas now we're transitioning from (dev.)data.solidarityeconomy.coop to (dev.)data.digitalcommons.coop that's not true. This base URL was set in the Apache config.

When you say 'hack' do you infer it is creating tech debt, or that you needed to do something non standard on the server? I think we may now always redirect from lod.coop to multiple domains?

wu-lee commented 8 months ago

Not sure, you could look at it either way...

To explain: the workers-coop data is put on dev.data.digitalcommons.coop, and pretty much everything else is still on dev.data.solidarityeconomy.coop. But will get moved across at some point.

lod.coop was configured such that only one base target URL was set; now we need to be able to have two so that workers-coops can be redirected to DCC, whilst the others stay the same.

It's a hack in that it wouldn't normally be necessary if we weren't transitioning from SEA to DCC domains, and everything could be in the same category. It's not a massive change, but I'll need to revert to that situation when the transition is fully done and all data is on DCC servers - unless for some reason we do need to spread our data across multiple domains (so far as I know we won't).

Note, we have another category of redirect for ESSGLOBAL links. This hasn't changed. We could have more, but we don't currently redirect anything else.

ColmDC commented 8 months ago

I can confirm map at

I've given the remaining subtasks above
tickboxes.

ColmDC commented 8 months ago

What update trigger are we using for this data set?

wu-lee commented 8 months ago

Just me on the keyboard currently

ColmDC commented 8 months ago

Just me on the keyboard currently

I recall they had a low expection of frequency of updates, but is there any reason we wouldn't do it next to live?

wu-lee commented 8 months ago

By default it would be updated every 10 mins, same as the limesurvey maps. W.C.'s feed doesn't show any info about the last changes.

ColmDC commented 6 months ago

Progress blocked by third party inactivity, so moving into backlog until unblocked.

wu-lee commented 6 months ago

The data feed from workers.coop has now been updated by Graham. I've gone through the old CSV field descriptions I had and compared with the new, amending where necessary.

Some questions which came out of that process, with Graham's answers.

Answers in and amongst below:

On 26 Feb 2024, at 17:47, Nick Stokoe nick.stokoe@code-operative.co.uk wrote:

Some questions which came out of that process:

How is the CSV encoded - e.g. UTF-8?

Yes, that’s my understanding.

Ownership type can be "Self employed" or "Workers" - is that correct? Doesn't seem to make sense for a type of ownership - more of a type of member.

I don’t disagree that it feels a bit weird, but I believe it is correct.

Some taxonomies include an "Unknown" value as well as empties in the actual data.
    Should empties be interpreted as "Unknown" or something else?
    Or perhaps an empty value should be flagged as a malformed value?
    Does this vary from field to field?

I created the ‘unknown’ value so that for records where we have no other value, we can explicitly tag them as unknown. Makes it easier to find them. but I haven’t yet set the ‘unknown’ value in place for all relevant records. So, empties are effectively the same as unknown, and when I find a bit of time they will all become unknown. Does it vary? My aim is to be consistent, so if you spot holes, please let me know.

There's a taxonomy for "CUK member status" but I can't see a field for that.

I think I left this field out because it is data that was imported from Cooperatives UK some time ago, is not actively managed by workers.coop as far as I’m aware, and is therefore pretty likely to be out of date now. Also I don’t think it is data that is in the public domain (i.e. it is internal to Cooperatives UK and i’m guessing was shared as a one-off with John. I’ll clarify with John.

I assume that organisations' Descriptions fields are always unformatted text, no paragraphs, correct?

I don’t know where the field content came from - I’m guessing from online forms, so people are just entering text into a textarea field. Some might put in two line breaks to create a gap between sentences. As far as I'm aware workers.coop makes no changes to this content. I’ve made a few edits to some of the entries because they included field delimiter characters that was breaking the CSV export.

Are these values of Description valid? I'm wondering as they're amongst the most common, but seem a bit devoid of information, wondering if it's mismatched or left-over data.
    "Local markets" (28x)
    "Local market" (6x)
    "A workers co-operative." (5x)

No idea.

wu-lee commented 6 months ago

The data conversion alterations for the aforementioned changes are now done, and the new dataset deployed here:

https://dev.data.digitalcommons.coop/workers-coop/ https://dev.data.digitalcommons.coop/workers-coop/standard.csv

The map here will show that data now, but the pop-up fields still need to be added.

https://dev.maps.coop/workers-coop/

Next step is https://github.com/DigitalCommons/mykomap/issues/233

wu-lee commented 5 months ago

On this point:

We probably need to know if we should filter out any rows by (say) status - currently Registered Status of "Dead" is omitted, historical reasons, nothing else.

Graham Mitchell suggests:

My opinion - don’t show orgs that are not actively trading, so I would only show ‘Live’ records.

This is what we currently do...

But John Atherton suggests:

We want the ability for people to see "non-live" records on the map, so they can let us know their status as this is one of the main data errors we have.

But, the default view on the map, should be to only show the lives ones, I assume it is possible though to filter by different things and have one set as the default?

This isn't possible in Mykomap - yet.

ColmDC commented 5 months ago

This isn't possible in Mykomap - yet.

We need to implement https://github.com/DigitalCommons/mykomap/issues/205

ColmDC commented 5 months ago

@wu-lee just checking there is no reason this isn't in In Progress? Moving it there as I'm pretty sure it is active.

wu-lee commented 5 months ago

It is active.

wu-lee commented 5 months ago

See #239 for final touches on this,

ColmDC commented 5 months ago

We can definitley consider a draft is in place, so closing this ticket and https://github.com/DigitalCommons/mykomap/issues/239 covers final touches on this,