Does Jobe support multiple files?

[cmenginnz]

Does Jobe support multiple files? In some cases it would be nice to upload both .c and .h files to compile.

[trampgeek]

I'm not quite sure if you're specifically asking about Jobe or more generally about CodeRunner.

Jobe supports upload of any number of files, which get cached in the file cache and are loaded into the working directory for the run. See here for the protocol.

CodeRunner has allowed question authors to attach multiple support files to questions for years. The most recent version of CodeRunner has an Attachments capability that lets students attach files to their submissions, too. The current Master branch version describes the feature as Experimental and has a couple of bugs but they have fixed in the Development branch. I intend to remove the "Experimental" label when I merge the Development into Master, which will should happen within the next month.

[cmenginnz]

My question is just in the scope of Jobe.

Let me elaborate on my question: I have to run a c project which includes multiple source files in multiple directories. For example, a project include such files: src/main.c src/lib/string.c src/lib/array.c src/header/string.h src/header/array.h

Does Jobe support to upload all these source files to run?

[trampgeek]

Certainly you get get Jobe to run a job of that sort but it requires a bit of ingenuity.

The first problem is that the default C-language task is set up to simply compile and run one file. #including of .h files is no problem provided they're in the same directory; simply attach them as support files and the #include "blah.h" will include them as normal. However, you have multiple .c files. A workaround here would be to set the task's compileargs parameter to list all the additional .c files, which, like the .h files, would be attached as support files. For example, provided you were happy to have all your .c and .h files in the same directory, your main.c would be the sourcecode for the job with compileargs set to (say) ['-Wall', 'string.c', 'array.c']. I should mention I've not actually done this, but I can't see any reason why it shouldn't work.

The second problem is that you've got files within subdirectories. That isn't directly possible as Jobe doesn't support a file hierarchy within the run-time directory. To solve that, I would write a program in Python, which would then be the main task to be executed, with all the other files as support files. The Python program would make the required directory structure, move the support files into their appropriate places, and then compile and run the C program. I do this sort of thing regularly but you do need to know how to write Python scripts that use the subprocess module. A variant of this is to have a Python script as the main program with all other files in a single .zip support file that has the appropriate hierarchy within it. The Python task then just unzips the support file and does a compile-and-execute command sequence.

[cmenginnz]

Supporting multiple source files within subdirectories via 'support files' does not work in my scenario. In my understanding, 'support files' are shared among all users. User A's support files maybe override user B's support file if they have the same name. In my scenario, there are a lot of students working on the same project and uploading their own modified copies to Jobe to compile and run. In this case, the support files are very likely got override.

Regarding having a Python task which unzips support file and does a compile-execute command, it also hits the above issue. And the Python task has to do the duplicated compile-execute jobs which Jobe has implemented already. And it gets more complicated while multiple languages are supported.

I have a rough idea about natively supporting multiple source files: Add two field src_tree and main_src_file in run_spec parameter. For example:

run_spec {
  "src_tree": {
    "src/main.c": "#include 'header/array.h'; ...",
    "src/header/array.h": "code of array.h",
    "src/lib/array.c": "code of array.c",
  },

  "main_src_file": ""src/main.c",
}

src_tree is an object, of which key is the full path of a source file, and value is the content of the source file. main_src_file specifies the main entry file if neccesary for some languages.

Jobe can dump all files into the run-time directory, then compile and execute it.

This is just my rough thought. Please feel free to have any comments.

I'm also willing to try to have an implement if you are happy with it.

[trampgeek]

Support files are not actually shared among users. If a job needs support files, its run-spec lists how the (unique) file identifiers should be mapped to the name given to the file within the private working directory for the run. Files are copied from the file cache into the working directory and given the specified name.

To load files into the cache, the API defines both a put_file and a post_file request, but only put_file is currently implemented. As used by CodeRunner, the file id supplied with each put_file request is the MD5 checksum of the file contents. This ensures (with an astronomically small but not mathematically zero risk) uniqueness of the file identifier. Multiple users can all upload their own versions of prog.c, prog.h etc but all will have unique file identifiers. This assumes there is at least a thin layer of software between the user and the Jobe API that takes users' files and computes their identifiers (MD5 checksums).

So I don't believe there is any real risk of students' files getting overridden.

Thank you for the offer to contribute, but I don't think there is a need for the changes you are suggesting unless I am still misunderstanding your use case.

What you could however add, if you would like and if you have a need, is the currently unimplemented post_file request. It could simply do an MD5 checksum of the contents and call (internally) the code for put_file, returning the MD5 checksum as the file identifier. Another extension that might help your use-case is to extend the meaning of the "file_name" string in the run-spec's file_list to a (relative) file path, e.g. "src/lib/array.c". This would simply involve creating the specified intermediate directories when copying a file from the cache into the user's working directory.