Bug Loading Context Directory with Complex File Types

[mfleader]

Describe the bug

An error occurs when the engine attempts to load a context directory that contains files that are not directories and not readable. For example, a file that is a symlink (soft symbolic link) to a directory is both not a directory and not readable. If this file is in the context directory, then the engine will throw an error, fail to load the context directory, and cease execution.

❯ ./arcaflow -context /tmp -config /tmp/config.yaml -input /tmp/input.yaml
2024-01-03T11:30:36-05:00    error    source=main    
Failed to load configuration file /tmp/config.yaml 
(failed to read file from context directory: /tmp/arcaflow-plugin-wait_latest/venv/lib64 
(read /tmp/arcaflow-plugin-wait_latest/venv/lib64: is a directory)) 

It cannot read the file lib64. lib64's mode bit is l which denotes that it is a symbolic link, and lib64 -> lib tells us that lib64 is a symbolic link that points to the directory lib.

❯ ls /tmp/arcaflow-plugin-wait_latest/venv/ -alh
total 8.0K
drwxr-xr-x. 1 mleader mleader  56 Jan  3 11:55 .
drwxr-xr-x. 1 mleader mleader   8 Jan  3 11:55 ..
drwxr-xr-x. 1 mleader mleader  20 Jan  3 11:55 lib
lrwxrwxrwx. 1 mleader mleader   3 Jan  3 11:55 lib64 -> lib 

To reproduce

arcaflow engine v0.9.0-beta3

/tmp/config.yaml

deployers:
  python:
    deployer_name: python
    modulePullPolicy: IfNotPresent
    pythonPath: python
    workdir: /tmp
log:
  level: debug
logged_outputs:
  error:
    level: error

/tmp/workflow.yaml

version: v0.2.0
input:
  root: RootObject
  objects:
    RootObject:
      id: RootObject
      properties: {}
steps:
  wait_1:
    plugin: 
      src: arcaflow-plugin-wait@git+https://github.com/arcalot/arcaflow-plugin-wait.git
      deployment_type: python
    step: wait
    input:
      seconds: 1    
outputs:
  a:
    b: !expr $.steps.wait_1.outputs 

/tmp/input.yaml

{}