Add forms UI

[bsaiken]

@WebDevJL I have decided how to implement the forms. We will add a left menu item for forms. This UI will look a lot like the services UI, so I started to prepare it, but then I realized the Services UI is not up to date.

forms/listALL: Left column: "Forms for the current project"; Right column: "All available forms"

forms/showForm: We don't have the webform yet - we can talk about it at our next meeting.

Then, in the task UI: Forms tab, it will look like the services tab: Left column: "Forms needed for this task" : Right Column: "Available forms for this project"

You can remove the database table "inspection_question" as we will not be using this.

I realize this is multiple issues - I just wanted to get it on the board to get things started.

[FWAJL]

@bsaiken: Ok for the Forms UI. I created a milestone in git for the related tasks.

I'm not sure what is not up to date with the Service UI. Please give me more details.

The DB table will be removed in the next release.

I have briefly reviewed your code but since it is new stuff, I'll look in details later, waiting you finish.

Jérémie Litzler

On 16 January 2015 at 18:41, bsaiken notifications@github.com wrote:

I have decided how to implement the forms. We will add a left menu item for forms. This UI will look a lot like the services UI, so I started to prepare it, but then I realized the Services UI is not up to date.

forms/listALL: Left column: "Forms for the current project"; Right column: "All available forms"

forms/showForm: We don't have the webform yet - we can talk about it at our next meeting.

Then, in the task UI: Forms tab, it will look like the services tab: Left column: "Forms needed for this task" : Right Column: "Available forms for this project"

You can remove the database table "inspection_question" as we will not be using this.

I realize this is multiple issues - I just wanted to get it on the board to get things started.

— Reply to this email directly or view it on GitHub https://github.com/FWAJL/FieldWorkAssistantMVC/issues/403.

[WebDevJL]

@bsaiken : the commit you made is merged to dev branch. Is this ready to go? if so, set the proper status so I know what to do. Thank you.

[bsaiken]

@FWAJL I created a mockup of what the field forms UI will look like:

Google Drive ▸ FieldWorkManager ▸ Field Forms UI.pdf

So, what I did is outdated. After our discussion we can formlized this issue so we can move it forward.

[damirius]

@FWAJL @bsaiken

I see that we removed inspection_question table, are there plans for a new one? We need relation table between projects and forms, now since forms are documents we can make relation table between project table and document table, but I don't think that's the proper way to go since we need to store "unique identifiers" somewhere. As we can see in the "Field Forms UI.pdf" we have "FWM_L" and "FWM_T" as identifiers, plus we have readable name of the document. Our uploader is hashing all file names, so that's one more reason to get separate form table for this. We would still use document table to store the actual name and id of the linked form record.

[bsaiken]

@damirius @FWAJL The inspection_question table is deprecated and would not have done what you need. Any time you think you need a table to accomplish your work you need to request a new database table from Jeremie. Jeremie won't even let me touch the database!

A agree that to allow us to promote Forms, as we do with Technicians, Locations, Analytes, and Service Providers, we need the relation tables project_form and task_form.

With photos, we could get by without a relation table because the photos cannot be promoted - they are assigned to a Location or Technician - they can't get re-assigned or change relations.

Form handling is more complex than I have identified here, and the identifiers serve a different function than relating the forms to Projects or Tasks.

[FWAJL]

@damirius @bsaiken

Have a look here and tell me what you think: https://docs.google.com/document/d/1ODkZDFSnLXrQ_Fo5GLUQeMbNm-ItJvCh1RV6Tm5DRgc/

Thank you.

[damirius]

@FWAJL @bsaiken

So those "unique identifiers" are the records stored in form_usages table in this case? In example that Brian gave us in "Field Forms UI.pdf", "FWM_L" and "FWM_T" would be stored in value field of form_usages table and description would be just readable text with more information about that form category right?

We maybe do not need document_id since we will store form_id in the start of the document_value string. That said, it will be easier of course to have document_id stored in the form table too, but it isn't common practice to store relations in both tables like that. Now this isn't probably the place for that, but we could add new field to the document table named document_category_id instead of storing the id at the start of the filename string. Now this isn't such a big deal, but it's something to think about.

So form table should contain "generic" templates too, those that admin can upload and they will be shared with everyone. We differentiate them from other forms by looking at their form_category and form_category_id values, if they are NULL they are shared and visible to every project manager, if they have something else stored in those fields we know that they are linked to some project/location/task, right? I see that we are creating copies of "generic" template forms when project manager link them to the project/task/location. We should also make a copy of actual document record too then? Since, as it was mentioned, document_value contains ID of the form at the start of the stored string (filename). So changes on the original "generic" form by admin would not change anything with "instanced/copied" forms. So for example if admin changes the form_usage_id in the form table for some generic form, copied forms will still have old form_usage record linked to them. I know this shouldn't happen, since you shouldn't have same form as location type form and next day as task type form. Also changing the actual pdf document of generic forms shouldn't change a thing with copied forms in this case. So I guess we are good on that front, if that's the functionality that we want.

[bsaiken]

@damirius @FWAJL Form handling is more complex than I thought.
Jeremie's tables may trump my file name proposal (e.g. "FWM_L").

As though it is not complex enough, consider these two items related to forms: 1) I have been working on a SOW for the Active Task UI, and there is another form attribute we need to account for: we need to identify if a form is complete or incomplete. 2) Forms may be completed, or filled out, outside of the website framework. For example, suppose a Technician prints a paper copy of the form, fills it out, takes a photo of the form, and uploads the form as an image. We will need to be able to identify the form. We can give the Field Technician a couple of prompt boxes to identify the Task, the form name and Location (if it is a location-specific form). Then we can covert the image to a pdf and create the proper file name to store the form.

Let me describe how I see forms being used here to make sure we cover everything:

Forms start as a template. Templates are provided by the web administrator and are fillable pdf forms. Templates are useable by any user. I have given Damir three sample fillable pdf forms.

If a user has their own pdf form, and they often will, they may request conversion to a fillable pdf form. I will create the fillable pdf template. The only difference is, since it originated from a user, that template is only related to that user's Projects. On the form UI page (Forms > List All), these user-specific templates will be pre-promoted and appear (only) on the left column.

Template forms can be viewed by the Project manager, but they would not be available to the Field Technicians.

We will use the templates to create the forms needed for each Task. There are two types: task-specific and location-specific. Location-specific forms are completed at each individual Location. So, if a Facility has 24 Locations, there will be 24 location-specific forms.
Task-specific forms have multiple rows, one row for each Location. There may not be enough rows on a single task-specific form for all the task Locations, so that form will need to have multiple pages, and each page should be a separate pdf file. We will need to write the code to generate the forms (Issue #672).

When a Field Technician is at a Location, they will click a button on their UI to pull up the required form. When they are finished, they will "submit" the form. Location-specific forms that are submitted are, by default, completed forms. Task-specific forms will have multiple submit events and are not complete until all the Locations on that form have been submitted.

The best way to complete the form is access the fillable pdf through the Technician UI on a mobile device, complete the form and submit it. This will make the form entries more legible (they would be keystrokes, not hand-written) and the data form entries would be extractable. We could even re-populate the data fields in task-specific forms so that in the final versions entries are stored in alphabetical order by Location name no matter what order the Field Technicians submitted the forms in.

I want to be able to accommodate other ways to complete forms, as in #2, above. We should be able to accommodate many form submission methods as long as we can create a unique identifier for each form that corresponds to our filing method.

Project Managers will be able to monitor the progress of a Task using an "Active Task UI". One of the tabs in the Active Task UI will be "Forms". This will be a dual list with incomplete forms on the right and completed forms on the left. The Project Manager will be able to view both completed and incomplete forms.

Once all the Task forms have been completed, the entire Task is complete and everything is archived. The archived tasks, including all the completed forms, can be viewed using the Left Menu item "Completed Tasks".

[FWAJL]

@damirius

1. Yes, form_usage_desc is like documentation so understand the purpose what we did later on.
2. In fact, you're right: document_id in the form table not necessary.
3. Yes to your question about form_category.
4. We will do copies of generic templates and project templates forms where creating task or location specific forms. It will add a row in form table and document table. If the generic or project form changes, I don't see why the specific forms would also. Especially if they are being edited or have been edited. @bsaiken : what do you think?

[bsaiken]

@FWAJL @damirius I am looking at this now. In order for me to respond, I am building a mock table set (document, form and form usage tables) for each possible document and form scenario. If you are making changes to the tables, as described above, please let me know so I can change my mock tables.

I have a question similar to Damir's. If you remove the document_id, because it is stored in the document_value, why do we need the document_content_type? It is also stored in the document_value.

If you want to see/edit my draft tables they are here, but they are very preliminary. https://docs.google.com/document/d/1BvEPbsmiZq5AXTUidCSxbuCis78yDBpPqkR8agkDyJk/edit

[bsaiken]

@FWAJL @damirius I do not see how, in the present scenario, you will be identify where completed forms belong. All child forms must be associated with a task_id, even location-specific forms.

[damirius]

@FWAJL @bsaiken

We still need content_type in the document table since we use it for other stuff as well, like photo uploads for locations. One big question we have here is if the user will be able to upload multiple pdfs for the same form. Brian mentioned this: "Task-specific forms have multiple rows, one row for each Location. There may not be enough rows on a single task-specific form for all the task Locations, so that form will need to have multiple pages, and each page should be a separate pdf file."

So do we just make separate form record for each pdf file, or do we upload multiple files and link them to the same form?

[bsaiken]

@damirius @FWAJL

Jeremie was pointing out that what we are doing in these discussions is very complex. He's correct. While this is interesting and useful to discuss at this time, we are getting WAY out of scope for what I intended for this issue. Make sure you look at the slides in the document "Field Forms UI.pdf" I uploaded to eLance. There are three (front-end) parts as I originally outlined.

1) list the three forms I gave you in the right hand column. 2) be able to promote the forms to the left hand column (project forms).
3) allow the user to upload a form (pdf file) and list it on the right hand column.

Here's how to accomplish this (modify as required): Create a new table, master_form, which, for now, will be a copy of the document table. This table will be accessible only to the web administrator through the Admin UI. Since the Admin UI is not done yet, you will need to manually populate the table. The image below shows the entries in the master_form table.

Create a new table, user_form. This will be similar to the document table, except it will include the foreign key "pm_id". This is where we will store the form information when a user uploads their own form.

We do not yet have a place to store the forms. Store master forms at ../Web/uploads/masterForms; store user forms at ../Web/uploads/userForms.

The right hand column will display the contents of /both/ user_form (by pm_id) and master_form.

For the left hand column contents, you will need to create a relation table, project_form, so you can identify which document(s) were promoted. The relation table has three columns: project_id, master_form_id, user_form_id. This is similar to what we have done for all the other UIs.

That's it - no document type identification yet. No document handling yet. No child form creation yet. You will not need the form or form_usage tables Jeremie created.

When submitting this for function review, you will need give me a sql script for the tables you created (with contents) and tell me what folders you created.

[damirius]

@bsaiken @FWAJL

Since this changes a lot of things, let me ask you couple of quick questions.

When we promote template form (master form) to project form, do we copy it or we just link existing one through relation table. We talked about that already for the last version of the tables, but I need to know how we want to do it with this one. If we copy it, I will just copy all the fields from selected master_form and add them to the new record in the user_form table, plus I'll copy the the existing file to the other directory. I'll add new record to the relation table after that.

[bsaiken]

@damirius Promoting an item does not copy anything. It only changes the relation table so we can see it on the list. It's just like promoting a Technician, Location or Service. We never create a copy of a Technician.
Try to forget all the discussion we had about "copying" forms - that comes later.

[damirius]

@bsaiken @FWAJL

Heya. I've pushed new commit on the feature_damir branch you can check it out there. First things first. You can find new DB update in Database/Script/Updates dir, under the name 2015-03-14.sql. First 3 queries will create necessary tables and third one will populate master_form table with sample data. Brian you can use those 3 pdf forms that you've sent me and store them in Web/uploads/master_form directory. User forms will be stored in Web/uploads/user_form. You don't need to actually create user_form directory it will be created automatically when you add your first user form, but since we are populating master_form table by hand we need to create this one manually.

OK, moving on. Make sure you have some project selected as the current project. After that going to List Forms -> List All (form/listAll) you will see the master forms in the right panel. This panel will also contain user_form records when PM add them. You can add them by clicking Add template button, design and name of it can be changed of course. This will bring up showForm page with dropzone's drag&drop field with "Form title" text field. You will notice that there is no add button because dropzone is automatically sending file when you drag it on the field or select it from the file browser. If you leave the title field blank, upload filename will be stored as one. We hash the filename later so we don't run into problems with files with the same file path. Please note that I left extension check there, so you can only upload pdfs. If everything goes well, you'll be redirected back to the list. You should see your new template in the right side panel along with other master forms. If you right click on new forms you'll bring up the context menu, you'll see delete option. Clicking on it will delete the user_form along with file and any relations in the project_form relationship table that were linked with it, session will update as well. I didn't add edit option yet since I don't know if we need it, we can only change whole file or maybe caption/title. Also keep in mind that right clicking on master forms will not bring up context menu because PM shouldn't be able to delete them. You can change the current project, you should see all the user forms uploaded by the PM and all master forms. Of course when you change your current project, all the forms should be shown in the right side panel unless you already promoted some forms for it before. You can also loggin onto some other PM. Select the project and go to Form List->List All page. You will see all master forms but no user forms, since user forms are shared between projects of the same PM but not between other PMs.

I'll talk more about technical part of this commit now. Since we use documents again, I decided to work on file storage DALs. I've found some DocumentDal class which seemed right for that. If it wasn't I'll create a new one, this one was empty in any case. I've added two new data access methods and modified existing one selectMany. Two new ones are addWithFile and deleteWithFile, for addWithFile we send the manager object and the file. It will preform the required checks for extension and it will upload the file in the appropriate directory. Directory is named after objects class in lower case. Because of that our form directories are called "master_form" and "user_form". deleteWithFile accepts same params as regular delete but it will first try to delete the file if it can. selectMany will try to add webPath to the object if it can. One very important thing to note is that when we create the manager we must set rootDirectory and webDirectory directly by assigning value to DocumentDal members or through the setters. Something like this depending where we call the manager of course: $manager->setRootDirectory($this->app()->config()->get(\Library\Enums\AppSettingKeys::RootDocumentUpload)); $manager->setWebDirectory($this->app()->config()->get(\Library\Enums\AppSettingKeys::BaseUrl) . $this->app()->config()->get(\Library\Enums\AppSettingKeys::RootUploadsFolderPath)); We also need to implement IDocument interface to our entities. We also need to add two private members, webPath and absolutePath. You'll notice that I used setFilename,setContentType,setContentSize and similar getters. In this way DocumentDal doesn't need to know how exactly we named our fields in the database but can still set them up. When I implemented setFilename method to our form entities I just passed the param to the value member and that's it. If we have our filepath field named differently we will just use $this->random_field_name = $filename; and it will work and DocumentDal doesn't need to know any of this, that's the primary reason why I've decided to use the interface for this. You'll also noticeValidExtension` method, we can return array here with the ext strings or just false if we don't want to check extensions for this entity.

I also wanted to mention that I've changed group_list left/right module. No need to worry, I just added new data attribute data-object since I have different objects in the panels, both user forms and master forms and I need to tell them apart somehow.

This is pretty lengthy, I think I'll stop here. Feel free to ask me anything regarding this issue.

[bsaiken]

@damirius I checked this and it is working great! I want a small change. Use the categorized_list module to differentiate FWM template forms from user-uploaded template forms. This will also give a more consistent look to the site since we already have Services in categorized lists. Technicians will also eventually be in categorized lists.
We could use the "category" column to differentiate, but I understand we may need this later for another function. For now, just use the category headings "Master Forms" and "User Forms" and I will change this later.

[damirius]

@bsaiken

Great. I've switched it from group lists to categorized lists in the new commit. I had to do some modifications similar to one done to group lists since we are displaying different type of objects in the list.

[bsaiken]

@damirius @FWAJL I will want to display the user forms module first, but I am going ahead and sending this to Jeremie as that is an inconsequential change.

[bsaiken]

@damirius btw - I should be the one changing the status to "code review". Thanks

[damirius]

@bsaiken

That's a quick change, I can do that before you send it for code review? You want me to push new commit?

Also sorry about setting it to code review, wanted to select "function review" from list, my bad.

[bsaiken]

@damirius You have time to make the change. Jeremie won't be on the clock until Monday morning.

[damirius]

@bsaiken

Thanks. It's done. I'll start working on the next issue.

[FWAJL]

@damirius

Please add USE baiken_fwm_1 at the start of your DB scripts. That allows to run them directly onto the right DB.

Also, please use the Dao generator for Dao classes: https://github.com/FWAJL/SqlToDao. It is setup to run the FieldWorkManager DB and it is just generating the Dao classes in the output folder. Run it to generate the getters and setters properly. This is what we used until now for all the Dao classes. If you have specific stuff that aren't generated, merge it together (e.g. IDocument interface for ex and related code).

@bsaiken @damirius

The folders to upload the forms should be generated on execution, not manually, otherwise, the Document interface is not useful. Can you achieve this @damirius ?

Thank you.

[FWAJL]

@damirius

Can you move the interfaces into Library/Interfaces, please? You'll need to create the folder. Thank you.

[damirius]

@FWAJL

I'll move it to Library/Interfaces, not a problem. Directory for uploads of specific table is executed when the first form is added. As I said before, we are creating master_form directory manually because we are using few forms for testing and we didn't want to implement functionality to add new master forms for now. It's done like that so we can test it out. user_form directory is created automatically on the first upload of the user_form.

[damirius]

@FWAJL

IDocument is moved to Library/Interfaces.