Describe the bug
When we use fluent bit to monitor logs from UNC or mapped network drive, it's throwing errors after log files are detected.
Same configuration works perfectly for log files on the local server.
There is no access issue or file existence issue. We made sure windows user running fluent bit has access to the remote path and files, and the files are not removed.
The remote network share drive can be a shared folder from other VM server, or Azure Storage File share - both of them are having the error after file is detected as shown below.
To Reproduce
[INPUT]
Name tail
Z is mapped network drive where permission is no issue
Path Z:\My Docs\*.txt
Tag botlog
Parser bot_log_parser
Rubular link if applicable:
Example log message if applicable:
[2024/07/03 22:42:17] [ info] [fluent bit] version=3.0.7, commit=73e72bdf9a, pid=9284
[2024/07/03 22:42:17] [debug] [engine] coroutine stack size: 98302 bytes (96.0K)
[2024/07/03 22:42:17] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2024/07/03 22:42:17] [ info] [cmetrics] version=0.9.1
[2024/07/03 22:42:17] [ info] [ctraces ] version=0.5.1
[2024/07/03 22:42:17] [ info] [input:tail:tail.0] initializing
[2024/07/03 22:42:17] [ info] [input:tail:tail.0] storage_strategy='memory' (memory only)
[2024/07/03 22:42:17] [debug] [tail:tail.0] created event channels: read=820 write=824
[2024/07/03 22:42:17] [debug] [input:tail:tail.0] flb_tail_fs_stat_init() initializing stat tail input
[2024/07/03 22:42:17] [debug] [input:tail:tail.0] inode=9367503187713458176 with offset=2116 appended as Z:\My Docs\Log_7_4_2024.txt
[2024/07/03 22:42:17] [debug] [input:tail:tail.0] 1 new files found on path 'Z:\My Docs\Log_7_4_2024.txt'
[2024/07/03 22:42:17] [debug] [file:file.0] created event channels: read=844 write=848
[2024/07/03 22:42:17] [debug] [stdout:stdout.1] created event channels: read=880 write=884
[2024/07/03 22:42:17] [debug] [router] match rule tail.0:file.0
[2024/07/03 22:42:17] [debug] [router] match rule tail.0:stdout.1
[2024/07/03 22:42:17] [ info] [sp] stream processor started
[2024/07/03 22:42:17] [ info] [output:stdout:stdout.1] worker #0 started
[2024/07/03 22:42:17] [ info] [output:file:file.0] worker #0 started
[2024/07/03 22:42:17] [debug] [input:tail:tail.0] inode=9367503187713458176 file=Z:\My Docs\Log_7_4_2024.txt promote to TAIL_EVENT
[2024/07/03 22:42:17] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory
[2024/07/03 22:42:17] [debug] [input:tail:tail.0] [static files] processed 0b, done
[2024/07/03 22:42:20] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory
[2024/07/03 22:42:22] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory
[2024/07/03 22:42:25] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory
[2024/07/03 22:42:27] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory
[2024/07/03 22:42:30] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory
[2024/07/03 22:42:32] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory
[2024/07/03 22:42:35] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory
[2024/07/03 22:42:37] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory
[2024/07/03 22:42:40] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory
[2024/07/03 22:42:42] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory
[2024/07/03 22:42:42] [ warn] [engine] service will shutdown in max 5 seconds
[2024/07/03 22:42:42] [ info] [input] pausing tail.0
[2024/07/03 22:42:43] [ info] [engine] service has stopped (0 pending tasks)
[2024/07/03 22:42:43] [ info] [input] pausing tail.0
[2024/07/03 22:42:43] [ info] [output:file:file.0] thread worker #0 stopping...
[2024/07/03 22:42:43] [debug] [input:tail:tail.0] inode=9367503187713458176 removing file name Z:\My Docs\Log_7_4_2024.txt
[2024/07/03 22:42:43] [ info] [output:file:file.0] thread worker #0 stopped
[2024/07/03 22:42:43] [ info] [output:stdout:stdout.1] thread worker #0 stopping...
[2024/07/03 22:42:43] [ info] [output:stdout:stdout.1] thread worker #0 stopped
- Steps to reproduce the problem:
**Expected behavior**
When log files are detected, fluent bit should send the logs to required OUTPUT instead of throwing above error saying no such file or directory.
**Your Environment**
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used: fluent bit 3.0.7
* Operating System and version: Windows server 2019
If any further info is required, please let me know. Thank you.
Bug Report
Hello All,
Describe the bug When we use fluent bit to monitor logs from UNC or mapped network drive, it's throwing errors after log files are detected. Same configuration works perfectly for log files on the local server. There is no access issue or file existence issue. We made sure windows user running fluent bit has access to the remote path and files, and the files are not removed.
The remote network share drive can be a shared folder from other VM server, or Azure Storage File share - both of them are having the error after file is detected as shown below.
To Reproduce
[INPUT] Name tail
Z is mapped network drive where permission is no issue
[2024/07/03 22:42:17] [ info] [fluent bit] version=3.0.7, commit=73e72bdf9a, pid=9284 [2024/07/03 22:42:17] [debug] [engine] coroutine stack size: 98302 bytes (96.0K) [2024/07/03 22:42:17] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128 [2024/07/03 22:42:17] [ info] [cmetrics] version=0.9.1 [2024/07/03 22:42:17] [ info] [ctraces ] version=0.5.1 [2024/07/03 22:42:17] [ info] [input:tail:tail.0] initializing [2024/07/03 22:42:17] [ info] [input:tail:tail.0] storage_strategy='memory' (memory only) [2024/07/03 22:42:17] [debug] [tail:tail.0] created event channels: read=820 write=824 [2024/07/03 22:42:17] [debug] [input:tail:tail.0] flb_tail_fs_stat_init() initializing stat tail input [2024/07/03 22:42:17] [debug] [input:tail:tail.0] inode=9367503187713458176 with offset=2116 appended as Z:\My Docs\Log_7_4_2024.txt [2024/07/03 22:42:17] [debug] [input:tail:tail.0] 1 new files found on path 'Z:\My Docs\Log_7_4_2024.txt' [2024/07/03 22:42:17] [debug] [file:file.0] created event channels: read=844 write=848 [2024/07/03 22:42:17] [debug] [stdout:stdout.1] created event channels: read=880 write=884 [2024/07/03 22:42:17] [debug] [router] match rule tail.0:file.0 [2024/07/03 22:42:17] [debug] [router] match rule tail.0:stdout.1 [2024/07/03 22:42:17] [ info] [sp] stream processor started [2024/07/03 22:42:17] [ info] [output:stdout:stdout.1] worker #0 started [2024/07/03 22:42:17] [ info] [output:file:file.0] worker #0 started [2024/07/03 22:42:17] [debug] [input:tail:tail.0] inode=9367503187713458176 file=Z:\My Docs\Log_7_4_2024.txt promote to TAIL_EVENT [2024/07/03 22:42:17] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory [2024/07/03 22:42:17] [debug] [input:tail:tail.0] [static files] processed 0b, done [2024/07/03 22:42:20] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory [2024/07/03 22:42:22] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory [2024/07/03 22:42:25] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory [2024/07/03 22:42:27] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory [2024/07/03 22:42:30] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory [2024/07/03 22:42:32] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory [2024/07/03 22:42:35] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory [2024/07/03 22:42:37] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory [2024/07/03 22:42:40] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory [2024/07/03 22:42:42] [error] [D:\a\fluent-bit\fluent-bit\plugins\in_tail\tail_file.c:1639 errno=2] No such file or directory [2024/07/03 22:42:42] [ warn] [engine] service will shutdown in max 5 seconds [2024/07/03 22:42:42] [ info] [input] pausing tail.0 [2024/07/03 22:42:43] [ info] [engine] service has stopped (0 pending tasks) [2024/07/03 22:42:43] [ info] [input] pausing tail.0 [2024/07/03 22:42:43] [ info] [output:file:file.0] thread worker #0 stopping... [2024/07/03 22:42:43] [debug] [input:tail:tail.0] inode=9367503187713458176 removing file name Z:\My Docs\Log_7_4_2024.txt [2024/07/03 22:42:43] [ info] [output:file:file.0] thread worker #0 stopped [2024/07/03 22:42:43] [ info] [output:stdout:stdout.1] thread worker #0 stopping... [2024/07/03 22:42:43] [ info] [output:stdout:stdout.1] thread worker #0 stopped