Create wireframe for bulk CSV upload steps

cafootitt commented 4 years ago

https://marvelapp.com/prototype/6agej0e/screen/73081453

Steps:

User prepares CSV offline
User clicks on "Add inventory via CSV" button when ready to upload
On new screen, the user selects the csv file from their computer to upload and clicks upload button, user sees some indication that the system is validating the csv and they should not click any button or close the tab until it's done, upload button becomes greyed out, so it can't be clicked again.
OpenSRP web validates the csv and returns a result
If there are errors, the user is alerted that csv did not validate and they can view the list of errors in the csv, which should be listed by row number of the csv. At this point, the user can select a new file and do the upload again.
If there are no errors, the user sees a summary report of all inventory that will be created. They see a question that's asking them if they want to confirm the upload or cancel the upload.
If they cancel, they get a confirmation that no new inventory has been added and are redirected back to the inventory page.
If they confirm, they get a confirmation that xx inventory items have been added and are redirected back to the inventory page.

moshthepitt commented 4 years ago

@cafootitt does this start with step 0: user downloads CSV template?

Might be useful if we need to ensure the locations etc have accurate ids so that we don't have to figure out the hard way what the inventory relates to

cafootitt commented 4 years ago

We'll provide a CSV template for the user, but we won't include the step to first download the CSV template.

rowo commented 4 years ago

@cafootitt

Where would we provide the CSV template for the user? Would this be a manual thing every time the user plans to update the inventory, or would it live somewhere else, just not part of the inventory page?
Can you link to the proposed CSV file format for this file?

cafootitt commented 4 years ago

@rowo So we'll just have the csv template shared on the google drive folder for the project, so it won't be tied to what we're building at all.

Here is the csv file format:

service_point_name	service_point_id*	product_name	product_id*	quantity*	delivery_date*	unicef_section*	serial_number*	donor*	po_number*
Alarobia Ambatomanga CSB	89879388	Midwifery Kit	S990222	1	04/08/2020	Health		Gates	897
Alarobia Ambatomanga CSB	89879388	Motorcycle	S2873493	1	05/01/2015	Health	198273972888	GAVI	898
Alarobia Ambatomanga CSB	89879388	Amoxicillin	S2729878	50	05/05/2020	Health		GAVI	897
EPP Tsaratanana	77787328	Motorcycle	S2873493	1	10/01/2015	Education	19783397288	UNICEF	897
EPP Tsaratanana	77787328	Scientific Suitcase	S2783472	100	30/12/2019	Education	287394872	UNICEF	897

Please check out this issue for details of how the csv validation will work on the server side.

rowo commented 4 years ago

@cafootitt thanks. So if there's only a blank template file, but not a file with a list of the actual locations, does that mean we'll need an easy way to export a location list or for the user to grab the latest service point name and corresponding service point IDs? That could help with what @moshthepitt is concerned about.

If so, this might need to be made a requirement either in the inventory service point list (i.e. have the service point ID show up on the main list and detail), or be able to download the this from the OpenSRP Web location admin page.

cafootitt commented 4 years ago

@rowo Ok, I get the concern now. So we've previously taken this into account and it's described in the SOW document. For the first time setup (that I'll be doing), someone will provide an export of the service point name and Ids directly from the locations API. After that, the UNICEF team will look up the ID of the service point on the inventory page. I think it's not currently included in the latest wireframe, but I've made a note to include instructions in the github implementation issue to display the service point ID on that view as well. If you have a few spare minutes to update the wireframe, that would be great: https://marvelapp.com/prototype/6agej0e/screen/73081456

rowo commented 4 years ago

@cafootitt Great, just wanted to make sure all that previous thinking gets translated over.

The concern definitely is for cases beyond the initial setup. I think the ID should go on the main list page in a column since that would allow easy copy/pasting across a few service points (or all). We removed that at some point, but if it's not a problem to put it back in, that'd be good.

cafootitt commented 4 years ago

@rowo That makes sense to also put it on that page. Thanks.

rowo commented 4 years ago

@cafootitt I actually had another thought this morning that this project should also be tracking how it goes with OpenSRP Web Locations Admin since it may also live there.

DHIS 2 does have an "export CSV" on locations that can download a location unit CSV with all the metadata, though I'm not sure it's in scope for v1. So it's just the recommended user workflow might change when this actually goes live.

cafootitt commented 4 years ago

@rowo Ok, thanks, will do so.

Yea, we left location CSV export out of scope for this phase.

rowo commented 4 years ago

@cafootitt on OpenSRP Web, Benjamin said they are discussing whether it'll be in v1 in the beginning of the week.

rowo commented 3 years ago

Example errors returned:

"Total Number of Rows in the CSV ",3 "Rows processed ",0 " " Row Number,Reason of Failure 1,"[Product ID does not exist in product catalogue, Service point ID does not exist, Donor is not valid, PO Number should be a whole number]" 2,[Service point ID does not exist] 3,"[Product ID does not exist in product catalogue, Service point ID does not exist, UNICEF section is not valid, Donor is not valid]"

rowo commented 3 years ago

@cafootitt here's a stab at this interaction: https://marvelapp.com/prototype/f3igd3h/screen/75790371. The solution will be a bit different with the upload being in its own section and not originating from the inventory list. Steps 7 and 8 are not included yet.

A few questions:

I don't know what the output is on the summary of changes. Is it a list of every single new inventory item with product name, service point and quantity information? This could be a really long list, which implies pagination, so we'd have to figure that out a bit.
How long does it actually take to commit the changes to the inventory after confirming you want to do it?
I think a cancel would be a nice to have during upload and validation. Is there a reason that's not included? Maybe it's too short to even need a cancel or the spinner dialogs? Or maybe it's not possible to cancel out?

cafootitt commented 3 years ago

@rowo Thanks for this. Looks great so far.

To answer your first question, in my mind I'm seeing just a summary such as "XXX products added to XX service points". To see the full list, that would basically just replicate what's in the csv, so I wasn't thinking we should do that. What do you think?

I'll have to consult with the engineering team on your 2nd question.

I think a cancel would be nice, so I'm not against including it. I can point to similar processes in OpenMRS, for example, where there is no cancel. The user could in theory cancel the process by closing the tab or going back, even though the page says not to do that.

rowo commented 3 years ago

@cafootitt 1a. Re: summary: a straight up listing I thought would be easiest even if it is long because then it's just a pagination issue. If it's a calculation based on what's in the CSV, that seemed harder (even if what displays is shorter) — but if the team think it'll be quick and easy that's okay with me. I'm unclear what constitutes a product here though — in terms of what a product is.. a quantity, a product from the product list, or individual products per service point? Might have to discuss what's easiest with an engineer. 1b. As far as the value of showing information at this step.. it's mainly about instilling confidence in the system before committing the changes. Whatever's easiest to build that lets the user think "that sounds about right" is probably what to aim for (since no matter what they can't really check line by line or easily know aggregates).

Okay please let me know. I think that impacts how 7 and 8 are done. I don't remember where I read it, but I thought I heard the upload and validating were ~30 seconds.. do you know if is that correct?
There's no real back here since the upload/validation happens in a modal. If engineering-wise it's possible to kill the process in a modal and be on the page to select a new file to upload (without reloading the page), that would be great. Or if not, we can handle the upload in a new page perhaps — I don't know what's less risky and simplest to build but UX wise either is fine with me. Just need to verify with an engineer.

cafootitt commented 3 years ago

@rowo We discussed this in depth this morning.

1a. The easiest and preferred approach is to show a message like "X inventory items will be added. Do you want to proceed?" The number of inventory items equates to the number of rows in the CSV, which is already being returned via the API call, so this doesn't represent really any additional effort to display. To display the full list of inventory items would require additional effort, because it requires another API call is what I understand. Are you happy with this in relation to your feedback in 1b?

The time it takes will correlate to the number of inventory items being added, so it could take a few seconds to much longer, but we will be able to show a progress bar, so the user can estimate how much time is remaining. @p-netm is going to think a bit more about how the data processing will work and what the user should or should not be doing during the processing. We'll discuss further on Friday's standup.
We didn't have time to discuss this, so it's pushed to Friday as well.

rowo commented 3 years ago

For 1b, Would a CSV of the below be equal to three inventory items? If so, only thing I can see is the definition of "inventory item" could be confusing to use here. Relatedly, it could make sense to use the same term chosen on this screen as well: (https://marvelapp.com/prototype/f3igd3h/screen/75791255), whatever it is, since they are the same value.

Alarobia Ambatomanga CSB, Amoxicillin, Quantity: 50 EPP Tsaratanana, Motorcycle, Quantity: 1 EPP Tsaratanana, Amoxicillin, Quantity: 200

cafootitt commented 3 years ago

@rowo Yes, correct, that would be 3 inventory items.

We can perhaps label it as "inventory line items" or "inventory row items". I think this is the easiest thing to reference, as it matches how we define an inventory item in our github libraries and one inventory item = 1 row in the CSV, so it's a quick check for the end user to do as well.

rowo commented 3 years ago

I updated the screens: https://marvelapp.com/prototype/f3igd3h/screens Inventory items seems to work the best seeing it there, but take a look.

cafootitt commented 3 years ago

Create new issue with implementation steps, and Peter will post his implementation plan in that issue.

peterMuriuki commented 3 years ago

on 2. it most likely will take a few seconds for the processing to take place.

I have been thinking about this, and its plausible to use the UI as it is. we would rely heavily on the app's global state so that the user can access the bulk-upload-process pages back and forth without effecting any ongoing validatin/upload process.
Pieces that would probably be added are:
- mechanism to inform the user that validation/ uploading process is ongoing/done, probably via a toast message or a banner. the toast could also probably include a quick link to the respective page in the csv upload workflow.
- constraint preventiing user from closing the page, while there is a csv upload in progress. this is technically harder to do.

cafootitt commented 3 years ago

@p-netm cool, thanks for the input. If you do that banner message, could you also do a link back to the csv bulk upload page: https://marvelapp.com/prototype/f3igd3h/screen/75885167? Or is that only possible with a toaster message?

I like that the user can navigate away from the page (but stay on eusm web) during data processing and come back to it with a banner or toaster when it's finished.

I don't think it would be worth forcing the user to keep the page open. But I am curious, say the user does kill the tab/page before the processing is finished. What would happen?

rowo commented 3 years ago

If upload/validation is probably under a minute or two, I would put "allowing navigating away and showing a banner" firmly in the nice to have category if it's any extra work or unknown. I would not spend any time on a constraint preventing a user from closing the page.

The easy workaround for a user is opening a new browser tab to explore EUSM or other websites. A popup warning would be useful to prevent accidentally leaving.

peterMuriuki commented 3 years ago

hanks for the input. If you do that banner message, could you also do a link back to the csv bulk upload page: https://marvelapp.com/prototype/f3igd3h/screen/75885167? Or is that only possible with a toaster message?

the banner can also have a link

I don't think it would be worth forcing the user to keep the page open. But I am curious, say the user does kill the tab/page before the processing is finished. What would happen?

the ongoing request would be canceled, this would not have any undesirable effects during the validation process, since no data is committed to the API. I am however not sure how the API will handle a cancel during the committing stage i.e. after user confirms the summary report from the validation step

I would put "allowing navigating away and showing a banner" firmly in the nice to have category if it's any extra work or unknown

This would probably take an extra ~12 hours

The easy workaround for a user is opening a new browser tab to explore EUSM or other websites. A popup warning would be useful to prevent accidentally leaving.

This is quickest and most straightforward implementation, I can start with this, for the initial implementation and iterate on it later according to QA findings.

cafootitt commented 3 years ago

Closing this and creating implementation issue for this feature here: https://github.com/OpenSRP/web/issues/307

opensrp / opensrp-client-eusm

Create wireframe for bulk CSV upload steps #12