search

planetsig / ufo-reports

NUFORC geolocated and time standardized ufo reports for close to a century of data. 80,000 plus reports.
132 stars 43 forks source link

'Comments' column - entries truncated #3

Open more-ginger opened 3 years ago

more-ginger commented 3 years ago

Hey there! First of all: thanks for the data, I am working to some visualizations and I am really enjoying the level of detail. I noticed that the comments column is often truncated, e.g.:

we were in the car and i looked out the window and i seen a unually big ball of a deep yellow color (way bigger than a shooting star li

I was wondering whether you knew what the problem is here: is it due to the way the data were entered?

Thank you very much for the help in advance

planetsig commented 3 years ago

fm,

The comments were there for verification. In other words one can go to the NUFORC site and look up the case and get all the details (maybe). The whole comments may have not been available and truncated on the site. I grabbed everything that was available. I would have to check yet knowing NUFORC data the person may have just kept typing and the web app truncated the rest of it. Or sometimes the ( and other punctuation will cause a truncation.

The focus was on geolocation and proximity with the missing persons records.

Have you seen the lone comment "mommy there here " OMG

Thanks for contacting me, Sig

On Mon, Dec 28, 2020 at 12:20 PM fmorini notifications@github.com wrote:

Hey there! First of all: thanks for the data, I am working to some visualizations and I am really enjoying the level of detail. I noticed that the comments column is often truncated, e.g.:

we were in the car and i looked out the window and i seen a unually big ball of a deep yellow color (way bigger than a shooting star li

I was wondering whether you knew what the problem is here: is it due to the way the data were entered?

Thank you very much for the help in advance

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/planetsig/ufo-reports/issues/3, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4UC55ERW7OA4B7U4SNQTLSXDEABANCNFSM4VMKMLPQ .

more-ginger commented 3 years ago

I see! Thanks for the direction. Checking on the NUFORC I can tell it's a scraping problem, it could even be fairly simple to fix! And yes.. some of them are really creepy, that's why it's so interesting.

Best, Francesca

planetsig commented 3 years ago

Francesca,

Yes, a simple fix. Good note. I believe there was a space issue on that machine at that time as well. Much better tools now too.

The missing persons incidents really got to me. When I drilled down into the info one quickly realizes that these people just vanished. And it is hard to believe just 10K in 100 years. Reported and not very well at that. Plus the blatant corruption of the data on missing persons.

My estimate is a real number is upto six times that number as many go unreported. Found a few more sources for missing as well. Check out missing 411. Human trafficking data varies by state and is also marginal. There is definitely something hidden there. It may be cultural or sinister.

Be well, be safe, get a posse of friends, have fun. Sig.

On Mon, Jan 4, 2021 at 12:49 PM fmorini notifications@github.com wrote:

I see! Thanks for the direction. Checking on the NUFORC I can tell it's a scraping problem, it could even be fairly simple to fix! And yes.. some of them are really creepy, that's why it's so interesting.

Best, Francesca

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/planetsig/ufo-reports/issues/3#issuecomment-754148979, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4UC554HYSDL4EJXCWREUTSYIESNANCNFSM4VMKMLPQ .