Error with metadata.json file handling on Windows

[kheil117]

Subject of the issue

Even if not officially supported, I used to run Indexer and now Conduit on Windows. During Importer initialization, Conduit reads and overwrites metadata.json file using a temporary metadata.json.temp file:

1. create temp file metadata.json.temp
2. write metadata to temp file metadata.json.temp
3. rename temp file metadata.json.temp to metadata.json

An file access error occurs at step 3. when renaming metadata.json.temp to metadata.json as the temp file is still open.

Your environment

• Conduit v1.4.0
• Follow Mode Node importer / postgresql database exporter
• Windows

Steps to reproduce

1. Launch conduit with or without metadata.json file or round override

Expected behaviour

Handle metadata.json.temp access.

I have modified the encodeToFile function in metadata.go as follow to prevent this error, I am no Golang expert at all so it is a very dumb way at the moment:

func (s *state) encodeToFile(dataDir string) error {
    pipelineMetadataFilePath := metadataPath(dataDir)
    tempFilename := fmt.Sprintf("%s.temp", pipelineMetadataFilePath)
    file, err := os.Create(tempFilename)
    if err != nil {
        return fmt.Errorf("encodeMetadataToFile(): failed to create temp metadata file: %w", err)
    }
    // defer file.Close() // REMOVED
    err = json.NewEncoder(file).Encode(s)
    if err != nil {
        return fmt.Errorf("encodeMetadataToFile(): failed to write temp metadata: %w", err)
    }
    file.Close() //ADDED
    err = os.Rename(tempFilename, pipelineMetadataFilePath)
    if err != nil {
        return fmt.Errorf("encodeMetadataToFile(): failed to replace metadata file: %w", err)
    }
    return nil
}

Actual behaviour

Critical failure:

Exiting with error: pipeline init error: Pipeline.Init() failed to write metadata to file: encodeMetadataToFile(): failed to replace metadata file: rename e:\test\Conduit\Importer/metadata.json.temp e:\test\Conduit\Importer/metadata.json: The process cannot access the file because it is being used by another process..