mdsol / rwslib

Provide a (programmer) friendly client library to Rave Web Services (RWS).
MIT License
31 stars 13 forks source link

Biostat API ProjectMetaDataRequest(project_name) is missing environment parameter #139

Closed dandekar-abhay closed 1 year ago

dandekar-abhay commented 1 year ago

Dear Team,

I noticed that the API : ProjectMetaDataRequest(project_name) from Biostat is missing an environment parameter.

Explained here: https://rwslib.readthedocs.io/en/latest/biostats_gateway.html#projectmetadatarequest-project-name

How can we know that the metadata is coming in from which environment? AFAIK, we can have multiple amendments in Dev, while the UAT and PROD can remain intact.

Please let me know if I am missing anything here.

Thank you.

isparks commented 1 year ago

As I recall, a CRF Version can be published to multiple environments (TEST, UAT, PROD etc) and unless you have the Prod Clinical Views option set they will share the same clinical views - which means that all the versions that are published are medged together. If Version1 has Field1, Field2, Field3 and Version2 has Field1, Field2 and Field4 then CVs have Field1, Field2, Field3, Field4.

If you have separate prod views set up (so there are special clinical views which ONLY reflect the versions published to PROD and not all the other environments) then you would expect to see something like this:

>>> from rwslib.rws_requests.biostats_gateway import MetaDataRequest
>>> r = RWSConnection('https://innovate.mdsol.com', 'username', 'password')  #Authorization required
>>> simplestudy_csv_meta = r.send_request(ProjectMetaDataRequest('SIMPLESTUDY'))
>>> print(simplestudy_csv_meta)
projectname,viewname,ordinal,varname,vartype,varlength,varformat,varlabel
"SIMPLESTUDY","V_SIMPLESTUDY_ENROL","1","userid","num","8","10.","Internal id for the user"
"SIMPLESTUDY","V_SIMPLESTUDY_ENROL","2","projectid","num","8","10.","projectid"
"SIMPLESTUDY","V_SIMPLESTUDY_ENROL","3","project","char","255","$255.","project"
"SIMPLESTUDY","prod.V_SIMPLESTUDY_ENROL","1","userid","num","8","10.","Internal id for the user"
"SIMPLESTUDY","prod.V_SIMPLESTUDY_ENROL","2","projectid","num","8","10.","projectid"
"SIMPLESTUDY","prod.V_SIMPLESTUDY_ENROL","3","project","char","255","$255.","project"

Where the prod.V_PROJECTNAME_FORMOID is included as a possible view.

dandekar-abhay commented 1 year ago

Thanks @isparks, your answer does help. So, it also means that we may not require an environment parameter for this API.

But before moving a little further on this, I would like to understand how the PROD CVs and the non-PROD CVs are related.

  1. Is it possible that we have a PROD clinical view not present in DEV/UAT?
  2. Is it possible that we have a DEV/UAT clinical view not present in PROD?
  3. Is there any subset-superset kind of relation between the clinical views on DEV/UAT/PROD?
  4. Can the clinical views on DEV/UAT/PROD be totally different (technically)?

IMHO, the answers to above questions can determine if we really need any environment parameter.

Thank you so much for your time.

isparks commented 1 year ago

I am far from being an expert on Clinical Views, you may have to ask Medidata....

But Medidata has some documentation about Clinical Views:

The structure of production Clinical Views is influenced only by CRF versions that

 are the default version of the production environment,  are the default version of a production site,  or have one or more production environment Subjects on them.

This is different from the "standard" Clinical Views (non-production Clinical Views) that are influenced by all published CRF versions, regardless of whether they have any Subjects on them or are the default version of a site or an environment.

It also means that the production Clinical Views might end up having their structure changed if they are rebuilt after an eCRF migration. This would result from all Subjects being migrated off of a CRF version that had been previously pushed to production.

But you only get separate clinical views created for production if you have this configuration set:

image

I hope from this it is clear that if you DON'T have this checkbox set then PROD/DEV/UAT/TEST/XXXX environments all share the same clinical views and the fields/forms in them are a superset of all the definitions from all these environments

If you DO have the checkbox set then PROD clinical views are totally separate from DEV/UAT/TEST/XXXX environment. In that case PROD views could look totally different to DEV/TEST/... etc because it only contains forms/columns that are in use defined by the checklist from Medidata's documentation above:

 are the default version of the production environment,  are the default version of a production site,  or have one or more production environment Subjects on them.

glow-mdsol commented 1 year ago

Were you able to resolve your issue @dandekar-abhay? Ok to close the issue?

dandekar-abhay commented 1 year ago

@glow-mdsol, thanks for asking. And yes, @isparks justification was helpful. We can close this issue.