search

gfilicetti / gke-github-deployment

This repository is a full example of a full CI/CD pipeline using Github actions, terraform and other tech.
Apache License 2.0
3 stars 2 forks source link

/output/gke/e149bb85-a0ee-408a-a35f-9273b222fa0e/stats.txt: No such file or directory #51

Closed alanpoole closed 2 months ago

alanpoole commented 2 months ago

GKE Job transcoding-f629069a-5074-481b-ab3f-48087efc45b2

2024-08-19 12:30:54.339 CST
Mounting file system "gcs-third-being-433017-p9-gcp-test-output"...
2024-08-19 12:30:54.340 CST
File system has been successfully mounted.
2024-08-19 12:33:54.848 CST
### Start time: 2024-08-19 18:33:54,847
2024-08-19 12:33:54.848 CST
### Transcoding video-0.mp4
2024-08-19 12:33:54.848 CST
### Output Path gke/e149bb85-a0ee-408a-a35f-9273b222fa0e
2024-08-19 12:33:54.937 CST
./entrypoint.sh: line 28: /output/gke/e149bb85-a0ee-408a-a35f-9273b222fa0e/stats.txt: No such file or directory
2024-08-19 12:33:54.941 CST
./entrypoint.sh: line 29: /output/gke/e149bb85-a0ee-408a-a35f-9273b222fa0e/stats.txt: No such file or directory
2024-08-19 12:33:54.946 CST
./entrypoint.sh: line 30: /output/gke/e149bb85-a0ee-408a-a35f-9273b222fa0e/stats.txt: No such file or directory
2024-08-19 12:33:57.377 CST
ffmpeg version n6.0-32-gd4a7a6e7fa Copyright (c) 2000-2023 the FFmpeg developers
2024-08-19 12:33:57.377 CST
  built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
2024-08-19 12:33:57.377 CST
  configuration: --cc= --cxx= --prefix=/usr/local --libdir=/usr/local/lib64 --bindir=/usr/local/bin --enable-shared --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --extra-cflags=-w --enable-nonfree --enable-libass --enable-libfreetype --disable-xlib --disable-sdl2 --disable-hwaccels --disable-vaapi --enable-libvpx --enable-libx264 --enable-gpl --enable-libx265 --enable-libsvthevc --enable-libsvtav1 --enable-libdav1d --extra-libs='-lpthread -lm'
2024-08-19 12:33:57.527 CST
  libavutil      58.  2.100 / 58.  2.100
2024-08-19 12:33:57.527 CST
  libavcodec     60.  3.100 / 60.  3.100
2024-08-19 12:33:57.527 CST
  libavformat    60.  3.100 / 60.  3.100
2024-08-19 12:33:57.527 CST
  libavdevice    60.  1.100 / 60.  1.100
2024-08-19 12:33:57.527 CST
  libavfilter     9.  3.100 /  9.  3.100
2024-08-19 12:33:57.527 CST
  libswscale      7.  1.100 /  7.  1.100
2024-08-19 12:33:57.527 CST
  libswresample   4. 10.100 /  4. 10.100
2024-08-19 12:33:57.527 CST
  libpostproc    57.  1.100 / 57.  1.100
2024-08-19 12:33:58.323 CST
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/input/video-0.mp4':
alanpoole commented 2 months ago

Investigating still....

https://cloud.google.com/blog/products/containers-kubernetes/using-the-cloud-storage-fuse-csi-driver-with-kubernetes

Another major challenge of running FUSE drivers in sidecar containers is that the FUSE runtime requires elevated container privileges. This restriction makes the sidecar-based solution almost impossible for GKE Autopilot, because privileged containers are not permitted on Autopilot clusters.
alanpoole commented 2 months ago

Switching the global mounting option to --dir-only appears to be fixing it.

https://cloud.google.com/storage/docs/gcsfuse-cli

alanpoole commented 2 months ago

All my tests passed last night, so the fix works.