Mr0grog
commented
5 years ago I tried looking for EIS's of a controversial sewage treatment plant near my home on these various tools… [when searching our prototype,] I got results, but none of them relevant to what I actually wanted… I tried searching the relevant terms in a Google Search, and immediately found what I was looking for:
The first thing that jumped out at me here that the EIS in question is from 2003, and the EPA database, which is the only data we have in the prototype, only goes back to 2012. So it’s a given that this was never going to be findable in our search.
I think we’d need to pick a newer topic if we wanted to test the potential utility of this kind of solution. More importantly, though, I feel like searching by name of facility (which often corresponds directly to the name of the document) isn’t a great use here — any of these would be easy to find through other means if you knew the name, and I think what the authors of that paper were getting at (both from the paper and from our meeting with them) is something more like: “can I search for salmon to see projects affected salmon habitats and how that was documented?” (See also your note about subject-based rather than title-based search; that was the main reason to start with full-text indexing, IIRC. It gets you partway to this without any need to figure out what tags or keywords to apply to every document.)
Full-text search: did we already accomplish this in the prototype?
This is really the only major thing we accomplished in this prototype :)
Try searching for “herbicide,” for example. That’s not in any titles, but you’ll get plenty of results.
Does Google Dataset Search do this? Also can't tell.
That one is also unclear to me, but it certainly seems like it’s more oriented towards things like tabular data, which might be components of EISes, but EISes themselves.
It's possible that at the moment, our demo accesses more [databases than Google Dataset Search]
Well, our demo only searches one (the EPA database), so this is either a totally divergent set from Google’s Dataset Search or a subset.
Direct link to downloadable PDF (again, can we do this? Or are we limited here?)
@b5 and @rgardaphe can answer this better, but IIRC there were two issues:
- The epa.gov database doesn’t always have a document, and sometimes it has many (and not just PDFs) for a given assessment listing. We punted a little on knowing what to do in that case.
- There was some concern about where to best store a big pile of these docs, so we punted on storing and providing access to them at all.
So, if I recalled that right, there’s no serious reason we couldn’t, except in cases where there was no actual document for us to download in the first place.
Non-standard formatting/content of documents: Wentz notes that a lack of formal standards makes it challenging to compare across documents & check accountability
I don’t think we were thinking about this point in particular, BUT if we could reasonably parse text out of it, we could always provide reformatted copies of every document in one standard HTML template. That’s a little iffy though — if you can only read 90% of the content, it’s still useful in a search index in a way that it might not be if you took that extracted text and tried to make a document from it again. You might be missing critical parts for a human who is reading it narratively.
Issues we might be able to work on (is this feasible/interesting for us?)… Record removal
I think the really important point here is that there has to both be very strong external interest and serious funding for these. If we are stepping up to be a reliable provider of long term storage here, that’s a big commitment, and we have to be careful that we and the data don’t suddenly disappear. I think we’d love to do that, but that means it really needs some serious, stable funding. And the argument we’ll face then is: why do we deserve the funding to do that more than, say, Hathi trust or Northwestern University Libraries, who are already sort of doing this?
are we in a position to provide something Google Dataset Search is not?
I think, as I said earlier, this actually isn’t a problem Google Dataset Search is really even attempting (I could definitely be wrong). And you certainly aren’t wrong about the other issues. As we’ve all said in meetings, to go a whole lot farther requires some serious commitment of time and energy from many people, finding both clever and just dumb brute-force solutions to things like: tagging docs, acquiring & scanning hard copies, building relationships (and ETL pipelines) to get digital copies from other libraries and agencies, etc. etc.
titaniumbones
dcwalk
shapironick
Frijol
tl;dr: key questions at bottom
Hi! I'm looking at this project & its roadmapping. As part of this process, I took a look at the original inspiration, the 2016 Wentz Paper.
Problem statement, as outlined in the Wentz paper
From the Wentz paper, I looked at the problem statement and classified outlined challenges by what I think we can and cannot do.
It has been two years since the paper was written, so these key points are also contextualized vs. the newly released Google Dataset Search.
Issues that we can tackle:
Issues that we are not in a position to approach:
Issues we might be able to work on (is this feasible/interesting for us?)
The elephant-in-room question is: are we in a position to provide something Google Dataset Search is not?
Having played around (slightly) with Google Dataset Search, I’m not very impressed. It seems they are running into several of the issues Wentz notes that I don’t think we’re in a position to tackle:
Edit: I had a quick user study here, but on reflection have removed it as not valid & distracting from the central conversation. It now lives here just for documentation purposes
OK, so what are the key questions?
I think this is a really cool project. But I also think that we need wholehearted "yes" answers to both of the above questions if we are to pursue it further.
Curious to hear thoughts on this.