This is what I got with the permission issue. This is a file produced by RMarkdown and in a html format.
Zihao Li
01/03/2024, 14:49
✖ +0000 UTC 2024-03-01 14:44 21.19 errored target ch_cost_lookupError: 'costs_care_home-Mar_2024-01_Mar.html' exists in'/conf/hscdiip/SLF_Extracts/Tests' but is not writeable. OK. It is because ch_cost_lookup reports error, consequently no input for source_sc_care_home
The logic of our "patch" on the permission issue before works well. The logic is that changing to a file permission to "write and read for sourcedev users" after a file is created. The issue now is that we only apply this patch to write_file() which includes rds and parquet files. But for files in other formats, we have not applied this patch. Hence, that's why I got the permission issue with this html file format.
The conclusion is:
We need to apply the patch to the RMarkdown file.
We may also need to double check if there are any more files in formats other "rds" and "parquet" written to the disk.
Here is a list of file formats in which files are written to the disk.
[x] rds
[x] parquet
[x] csv.gz (only for compressing csv files in"{year}/Extracts/")
[x] excel (only for tests results)
[x] html (produced by RMarkdown)
Keep adding comments to this issue if one finds more files with other formats that need to be written to the disk, and I will modify the list above.
This is what I got with the permission issue. This is a file produced by RMarkdown and in a html format.
The logic of our "patch" on the permission issue before works well. The logic is that changing to a file permission to "write and read for sourcedev users" after a file is created. The issue now is that we only apply this patch to
write_file()which includesrdsandparquetfiles. But for files in other formats, we have not applied this patch. Hence, that's why I got the permission issue with this html file format.The conclusion is:
Here is a list of file formats in which files are written to the disk.
Keep adding comments to this issue if one finds more files with other formats that need to be written to the disk, and I will modify the list above.