potential module test results csv export

[lantone]

Hi @gneeser ,

After my accidental but successful DoS attack on the database this afternoon, Nick and I were discussing the options. I was running a script that loaded each module's results page and scraped out the relevant data for offline use. And it broke your internet. Sorry about that.

One identified alternative (among others) is to create a csv file with all desired information, analogous to the modulereport.csv already available: http://inky.physics.purdue.edu/cmsfpix//Submission_p/tmp/modulereport.csv

The relevant fields would be the following: module name grade 16x ROC failure mode (empty for good ROCs) date of reception date of shipping date of final judgement (status set to either "Rejected" or "Ready for Mounting")

This issue is mainly to document this potential solution until Nick has a chance to talk to you about the feasibility of implementing this.

Thanks,

Jamie

[gneeser]

Jamie, I talked with Nick and we think a .csv would be the best way to approach this. I'll get started on it and let you know when I have something. In the meantime, if you think of anything else, let me know.

-Greg

[gneeser]

One question just occurred to me - do you only want modules that have had a final decision made (i.e. modules that are rejected or ready for mounting / already mounted)?

-Greg

[lantone]

Good question - I’d love to have everything, with blank entries for things that don’t exist yet, e.g. “Judgement Date” for things still being processed.

On Jun 2, 2016, at 3:42 PM, gneeser notifications@github.com wrote:

One question just occurred to me - do you only want modules that have had a final decision made (i.e. modules that are rejected or ready for mounting / already mounted)?

-Greg

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/puhep/pudb/issues/184#issuecomment-223415831, or mute the thread https://github.com/notifications/unsubscribe/AFH89aWtUY2EoR-UA52YPYNcbqd2PYk_ks5qH0AigaJpZM4IsARC.

[gneeser]

Jamie, Here's what I have so far: http://inky.physics.purdue.edu/cmsfpix//Submission_t/summary/modulereport2.php?

It has just about everything you asked for with one large exception: there is no "Rejected" date recorded in the database, so the "Judged" column is only filled for modules that were set to Ready for Mounting.

This is a problem that has come up in other places in the database, so hopefully I'll have that retroactively added soon, as I'll be placing a higher priority on it.

Can you think of anything else that you'd like from this document in the meantime?

-Greg

[lantone]

So far so good, there’s nothing else I can think of for now, I’m ~80% sure I won’t think of anything else in the future either.

Thanks for making quick work of that!

Jamie

On Jun 3, 2016, at 2:55 PM, gneeser <notifications@github.com mailto:notifications@github.com> wrote:

Jamie, Here's what I have so far: http://inky.physics.purdue.edu/cmsfpix//Submission_t/summary/modulereport2.php http://inky.physics.purdue.edu/cmsfpix//Submission_t/summary/modulereport2.php?

It has just about everything you asked for with one large exception: there is no "Rejected" date recorded in the database, so the "Judged" column is only filled for modules that were set to Ready for Mounting.

This is a problem that has come up in other places in the database, so hopefully I'll have that retroactively added soon, as I'll be placing a higher priority on it.

Can you think of anything else that you'd like from this document in the meantime?

-Greg

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/puhep/pudb/issues/184#issuecomment-223678400, or mute the thread https://github.com/notifications/unsubscribe/AFH89TySboLHoLYmRuwN9d-tenpmO_PRks5qIIaZgaJpZM4IsARC.

[lantone]

Actually, the one thing I could conceive of adding would be the various other steps in production/testing, like "visually inspected", "HDI glued", and "test flags set to X", but for the moment those seem like overkill

[lantone]

I've having a bit of trouble parsing the new csv file, I think maybe you need to add commas at the end of each line (also supported by the format of the original csv[1])

Thanks, Jamie

[1] http://inky.physics.purdue.edu/cmsfpix//Submission_t/tmp/modulereport.csv

[lantone]

ok, the real problem is that there are two extra columns in each row with respect to the top line of column headers (one true extra column and one due to the lack of a comma in the header row).

[lantone]

and if it's not too much trouble it'd be nice to have the full date info (day hour:minute:sec) since i have that format by default elsewhere

[lantone]

another request: if a module is mounted, could we add its position on blade as a column? turns out it's tough to track all these guys down...

[gneeser]

Jamie, How does it work/look now? http://inky.physics.purdue.edu/cmsfpix//Submission_t/summary/modulereport2.php?

Let me know if you still have trouble parsing it.

-Greg

[lantone]

Hi Greg,

Looking good, only thing is there's still an extra column in every row wrt the first header row.

[gneeser]

I think I found it this time. Do you still see it?

[lantone]

looks like you got it, we're all set!

[gneeser]

Alright, sounds good. Let me know if you end up wanting any other features.

-Greg

[gneeser]

You know, it would probably help if I made a button so that you could get to it on Submission_p. I'll do that.

[gneeser]

Alright, it's on Submission_p on the main page as "Module Report 2." On a related note, if you'd like a more creative name than "Module Report 2," just let me know.

[lantone]

Hi @gneeser , Would it be feasible to add the IV and X-ray grades to this report? They're not listed on the module test result page explicitly unless they're less than A, so maybe it's not straightforward?

Thanks, Jamie

[gneeser]

I could definitely add those grades, it wouldn't be terribly complex.

[lantone]

It also appears the csv isn't updating. The most recent date listed is 2016-06-24 13:47:34... I'm pretty sure we've done some stuff since then :-)

[NickHinton]

I checked earlier today and it appears to be working now, with no clear indication of why it wasn't behaving before.

[lantone]

hi greg,

while you're adding the X-ray and IV grades, could you also include a column with the "Total number bad/defective pixels"? it could be called NBadPix, or somesuch.

thanks,

jamie

[gneeser]

Jamie, Sorry about the delay - give it a try now. Let me know if you catch errors or discrepancies.

-Greg

[lantone]

Great, thanks Greg! Just FYI, I only see the updates in the test instance of the DB, not the production instance

[gneeser]

Are you loading the php or just looking at the file? The file is newly generated everytime that the php is loaded. I didn't run the php in submission_p, so the file wouldn't be updated.

[lantone]

I'm just loading the url to view it, i.e. pointing my browser here:

http://inky.physics.purdue.edu/cmsfpix//Submission_p/tmp/modulereport2.csv

Is there a way I should be accessing it so that it updates?

[gneeser]

http://inky.physics.purdue.edu/cmsfpix//Submission_p/summary/modulereport2.php?

[gneeser]

It's the "Module Report 2" button on the main page.

[lantone]

gotcha, thanks

[NickHinton]

@lantone Jamie, are you happy with this feature? Can we close the issue?

[lantone]

yep!