search

NVIDIA / gds-nvidia-fs

NVIDIA GPUDirect Storage Driver
Other
207 stars 31 forks source link

cuFileWriteAsync & cuFileReadAsync failed with bytes_written_p =-5 #46

Closed glingyan closed 3 months ago

glingyan commented 3 months ago

nvidia-fs log is below [ 938.530342] nvidia-fs:nvfs ioctl map success [ 938.540761] nvidia-fs:NVFS VMA close vma:00000000dd88f1d2 nvfs_mgroup 000000000b447889 [ 938.540768] nvidia-fs:munmap invoked - IO state TERMINATED 5 5 [ 938.540769] nvidia-fs:*****munmap invoked - nvfs_mgroup ref 2 mgroup 000000000b447889

Environment is below Redhat 9.0 + MLNX_OFED_LINUX-24.04-0.7.0.0-rhel9.0-x86_64 (nvidia-fs require IB driver) + cuda_12.6.0_560.28.03_linux.run

[root@localhost src]# cat /proc/driver/nvidia-fs/stats GDS Version: 1.10.0.4 NVFS statistics(ver: 4.0) NVFS Driver(version: 2.20.5) Mellanox PeerDirect Supported: True IO stats: Enabled, peer IO stats: Enabled Logging level: debug

Active Shadow-Buffer (MiB): 0 Active Process: 0 Batches : n=0 ok=0 err=0 Avg-Submit-Latency(usec)=0 Reads : n=0 ok=0 err=0 readMiB=0 io_state_err=0 Reads : Bandwidth(MiB/s)=0 Avg-Latency(usec)=0 Sparse Reads : n=0 io=0 holes=0 pages=0 Writes : n=0 ok=0 err=0 writeMiB=0 io_state_err=0 pg-cache=0 pg-cache-fail=0 pg-cache-eio=0 Writes : Bandwidth(MiB/s)=0 Avg-Latency(usec)=0 Mmap : n=1 ok=1 err=0 munmap=1 Bar1-map : n=1 ok=1 err=0 free=1 callbacks=0 active=0 delay-frees=0 Error : cpu-gpu-pages=0 sg-ext=0 dma-map=0 dma-ref=0 Ops : Read=0 Write=0 BatchIO=0 GPU 0000:8b:00.0 uuid:4964db7b-bd9e-ca34-cdb6-ebae142a2571 : Registered_MiB=0 Cache_MiB=0 max_pinned_MiB=1 cross_root_port(%)=0

glingyan commented 3 months ago

[ 612.463083] nvidia_fs: Initializing nvfs driver module [ 612.463090] nvidia_fs: registered correctly with major number 509 [ 612.463118] nvidia-fs:nvfs device count: 16 [ 612.463877] nvidia-fs:nvfs listing GPU paths: [ 612.463972] nvidia-fs:nvfs_pci pci device 0000:8b:00:0 width:16 speed_index:5 bw:80 [ 612.463975] nvidia-fs:nvfs_pci pci device entry[32] 0000:8b:00:0 path: [ 612.463981] nvidia-fs:nvfs_pci bridge: 0000:8a:02:0 (acs=1/1) depth :0 [ 612.464085] nvidia-fs:nvfs_pci pci device 0000:0e:00:0 width:0 speed_index:0 bw:0 [ 612.464142] nvidia-fs:nvfs listing IB paths: [ 612.464223] nvidia-fs:nvfs_pci pci device 0000:0c:00:0 width:1 speed_index:1 bw:1 [ 612.464224] nvidia-fs:nvfs_pci pci device entry[39] 0000:0c:00:0 path: [ 612.464229] nvidia-fs:nvfs_pci bridge: 0000:0a:07:0 (acs=1/1) depth :0 [ 612.464243] nvidia-fs:nvfs_pci pci device 0000:20:00:0 width:16 speed_index:5 bw:80 [ 612.464244] nvidia-fs:nvfs_pci pci device entry[57] 0000:20:00:0 path: [ 612.464248] nvidia-fs:nvfs_pci bridge: 0000:1f:02:0 (acs=1/1) depth :0 [ 612.464291] nvidia-fs:nvfs_pci pci device 0000:b4:00:0 width:16 speed_index:5 bw:80 [ 612.464292] nvidia-fs:nvfs_pci pci device entry[22] 0000:b4:00:0 path: [ 612.464297] nvidia-fs:nvfs_pci bridge: 0000:b3:02:0 (acs=1/1) depth :0 [ 612.464326] nvidia-fs:nvfs listing NVME paths: [ 612.464330] nvidia-fs:nvfs_pci pci device 0000:0b:00:0 width:2 speed_index:4 bw:8 [ 612.464331] nvidia-fs:nvfs_pci pci device entry[20] 0000:0b:00:0 path: [ 612.464335] nvidia-fs:nvfs_pci bridge: 0000:0a:06:0 (acs=1/1) depth :0 [ 612.464370] nvidia-fs:nvfs_pci pci device 0000:ea:00:0 width:2 speed_index:4 bw:8 [ 612.464371] nvidia-fs:nvfs_pci pci device entry[58] 0000:ea:00:0 path: [ 612.464377] nvidia-fs:nvfs_pci bridge: 0000:e9:02:0 (acs=1/1) depth :0 [ 612.464382] nvidia-fs:nvfs_pci pci device 0000:eb:00:0 width:4 speed_index:5 bw:20 [ 612.464383] nvidia-fs:nvfs_pci pci device entry[23] 0000:eb:00:0 path: [ 612.464388] nvidia-fs:nvfs_pci bridge: 0000:e9:04:0 (acs=1/1) depth :0 [ 612.464393] nvidia-fs:nvfs_pci pci device 0000:ec:00:0 width:4 speed_index:4 bw:16 [ 612.464393] nvidia-fs:nvfs_pci pci device entry[21] 0000:ec:00:0 path: [ 612.464399] nvidia-fs:nvfs_pci bridge: 0000:e9:06:0 (acs=1/1) depth :0 [ 612.464569] nvidia-fs:nvfs_pci: pci_dist matrix[gpu=32][peer=20] 0000:8b:00.0 ->0000:0b:00.0 pci_dist:257 [ 612.464632] nvidia-fs:nvfs_pci: pci_dist matrix[gpu=32][peer=21] 0000:8b:00.0 ->0000:ec:00.0 pci_dist:257 [ 612.464691] nvidia-fs:nvfs_pci: pci_dist matrix[gpu=32][peer=22] 0000:8b:00.0 ->0000:b4:00.0 pci_dist:130 [ 612.464743] nvidia-fs:nvfs_pci: pci_dist matrix[gpu=32][peer=23] 0000:8b:00.0 ->0000:eb:00.0 pci_dist:257 [ 612.464771] nvidia-fs:nvfs_pci: pci_dist matrix[gpu=32][peer=39] 0000:8b:00.0 ->0000:0c:00.0 pci_dist:257 [ 612.464800] nvidia-fs:nvfs_pci: pci_dist matrix[gpu=32][peer=57] 0000:8b:00.0 ->0000:20:00.0 pci_dist:257 [ 612.464852] nvidia-fs:nvfs_pci: pci_dist matrix[gpu=32][peer=58] 0000:8b:00.0 ->0000:ea:00.0 pci_dist:257 [ 938.216833] nvidia-uvm: Loaded the UVM driver, major device number 503. [ 938.522032] nvidia-fs:nvfs_open 0 [ 938.522062] nvidia-fs:nvfs_open 0 [ 938.522082] nvidia-fs:nvfs_open 0 [ 938.522103] nvidia-fs:nvfs_open 0 [ 938.522122] nvidia-fs:nvfs_open 0 [ 938.522140] nvidia-fs:nvfs_open 0 [ 938.522158] nvidia-fs:nvfs_open 0 [ 938.522177] nvidia-fs:nvfs_open 0 [ 938.522195] nvidia-fs:nvfs_open 0 [ 938.522214] nvidia-fs:nvfs_open 0 [ 938.522232] nvidia-fs:nvfs_open 0 [ 938.522250] nvidia-fs:nvfs_open 0 [ 938.522268] nvidia-fs:nvfs_open 0 [ 938.522286] nvidia-fs:nvfs_open 0 [ 938.522305] nvidia-fs:nvfs_open 0 [ 938.522323] nvidia-fs:nvfs_open 0 [ 938.529414] nvidia-fs:mmap 00000000dd88f1d2, file:0000000030458933 [ 938.529419] nvidia-fs:mmap request for file: nvidia-fs0 [ 938.529422] nvidia-fs:base_index 164a79fe8 not found [ 938.529426] nvidia-fs:Assigning nvfs_mgroup 000000000b447889 to vma 00000000dd88f1d2 [ 938.529430] nvidia-fs:vm_insert_page : 0 pages: ff919d80850f9540 mapping: 0000000000000000, index: 164a79fe8000 (7f5b596bd000 - 7f5b596be000) ret: 0
[ 938.529432] nvidia-fs:vm_insert_page : 1 pages: ff919d80850f95c0 mapping: 0000000000000000, index: 164a79fe8001 (7f5b596be000 - 7f5b596bf000) ret: 0
[ 938.529435] nvidia-fs:vm_insert_page : 2 pages: ff919d80850f9600 mapping: 0000000000000000, index: 164a79fe8002 (7f5b596bf000 - 7f5b596c0000) ret: 0
[ 938.529436] nvidia-fs:vm_insert_page : 3 pages: ff919d80850f9640 mapping: 0000000000000000, index: 164a79fe8003 (7f5b596c0000 - 7f5b596c1000) ret: 0
[ 938.529438] nvidia-fs:vm_insert_page : 4 pages: ff919d80850f9680 mapping: 0000000000000000, index: 164a79fe8004 (7f5b596c1000 - 7f5b596c2000) ret: 0
[ 938.529440] nvidia-fs:vm_insert_page : 5 pages: ff919d80850f96c0 mapping: 0000000000000000, index: 164a79fe8005 (7f5b596c2000 - 7f5b596c3000) ret: 0
[ 938.529441] nvidia-fs:vm_insert_page : 6 pages: ff919d80850f9700 mapping: 0000000000000000, index: 164a79fe8006 (7f5b596c3000 - 7f5b596c4000) ret: 0
[ 938.529443] nvidia-fs:vm_insert_page : 7 pages: ff919d80850f9740 mapping: 0000000000000000, index: 164a79fe8007 (7f5b596c4000 - 7f5b596c5000) ret: 0
[ 938.529445] nvidia-fs:vm_insert_page : 8 pages: ff919d80850f9780 mapping: 0000000000000000, index: 164a79fe8008 (7f5b596c5000 - 7f5b596c6000) ret: 0
[ 938.529446] nvidia-fs:vm_insert_page : 9 pages: ff919d80850f97c0 mapping: 0000000000000000, index: 164a79fe8009 (7f5b596c6000 - 7f5b596c7000) ret: 0
[ 938.529448] nvidia-fs:vm_insert_page : 10 pages: ff919d80850f9800 mapping: 0000000000000000, index: 164a79fe800a (7f5b596c7000 - 7f5b596c8000) ret: 0
[ 938.529450] nvidia-fs:vm_insert_page : 11 pages: ff919d80850f9840 mapping: 0000000000000000, index: 164a79fe800b (7f5b596c8000 - 7f5b596c9000) ret: 0
[ 938.529451] nvidia-fs:vm_insert_page : 12 pages: ff919d80850f9880 mapping: 0000000000000000, index: 164a79fe800c (7f5b596c9000 - 7f5b596ca000) ret: 0
[ 938.529452] nvidia-fs:vm_insert_page : 13 pages: ff919d80850f98c0 mapping: 0000000000000000, index: 164a79fe800d (7f5b596ca000 - 7f5b596cb000) ret: 0
[ 938.529454] nvidia-fs:vm_insert_page : 14 pages: ff919d80850f9900 mapping: 0000000000000000, index: 164a79fe800e (7f5b596cb000 - 7f5b596cc000) ret: 0
[ 938.529456] nvidia-fs:vm_insert_page : 15 pages: ff919d80850f9940 mapping: 0000000000000000, index: 164a79fe800f (7f5b596cc000 - 7f5b596cd000) ret: 0
[ 938.529457] nvidia-fs:vm_insert_page : 16 pages: ff919d80850f9980 mapping: 0000000000000000, index: 164a79fe8010 (7f5b596cd000 - 7f5b596ce000) ret: 0
[ 938.529459] nvidia-fs:vm_insert_page : 17 pages: ff919d80850f99c0 mapping: 0000000000000000, index: 164a79fe8011 (7f5b596ce000 - 7f5b596cf000) ret: 0
[ 938.529460] nvidia-fs:vm_insert_page : 18 pages: ff919d80850f9a00 mapping: 0000000000000000, index: 164a79fe8012 (7f5b596cf000 - 7f5b596d0000) ret: 0
[ 938.529461] nvidia-fs:vm_insert_page : 19 pages: ff919d80850f9a40 mapping: 0000000000000000, index: 164a79fe8013 (7f5b596d0000 - 7f5b596d1000) ret: 0
[ 938.529463] nvidia-fs:vm_insert_page : 20 pages: ff919d80850f9a80 mapping: 0000000000000000, index: 164a79fe8014 (7f5b596d1000 - 7f5b596d2000) ret: 0
[ 938.529465] nvidia-fs:vm_insert_page : 21 pages: ff919d80850f9ac0 mapping: 0000000000000000, index: 164a79fe8015 (7f5b596d2000 - 7f5b596d3000) ret: 0
[ 938.529467] nvidia-fs:vm_insert_page : 22 pages: ff919d80850f9b00 mapping: 0000000000000000, index: 164a79fe8016 (7f5b596d3000 - 7f5b596d4000) ret: 0
[ 938.529469] nvidia-fs:vm_insert_page : 23 pages: ff919d80850f9b40 mapping: 0000000000000000, index: 164a79fe8017 (7f5b596d4000 - 7f5b596d5000) ret: 0
[ 938.529471] nvidia-fs:vm_insert_page : 24 pages: ff919d80850f9b80 mapping: 0000000000000000, index: 164a79fe8018 (7f5b596d5000 - 7f5b596d6000) ret: 0
[ 938.529472] nvidia-fs:vm_insert_page : 25 pages: ff919d80850f9bc0 mapping: 0000000000000000, index: 164a79fe8019 (7f5b596d6000 - 7f5b596d7000) ret: 0
[ 938.529473] nvidia-fs:vm_insert_page : 26 pages: ff919d80850f9c00 mapping: 0000000000000000, index: 164a79fe801a (7f5b596d7000 - 7f5b596d8000) ret: 0
[ 938.529475] nvidia-fs:vm_insert_page : 27 pages: ff919d80850f9c40 mapping: 0000000000000000, index: 164a79fe801b (7f5b596d8000 - 7f5b596d9000) ret: 0
[ 938.529476] nvidia-fs:vm_insert_page : 28 pages: ff919d80850f9c80 mapping: 0000000000000000, index: 164a79fe801c (7f5b596d9000 - 7f5b596da000) ret: 0
[ 938.529478] nvidia-fs:vm_insert_page : 29 pages: ff919d80850f9cc0 mapping: 0000000000000000, index: 164a79fe801d (7f5b596da000 - 7f5b596db000) ret: 0
[ 938.529479] nvidia-fs:vm_insert_page : 30 pages: ff919d80850f9d00 mapping: 0000000000000000, index: 164a79fe801e (7f5b596db000 - 7f5b596dc000) ret: 0
[ 938.529481] nvidia-fs:vm_insert_page : 31 pages: ff919d80850f9d40 mapping: 0000000000000000, index: 164a79fe801f (7f5b596dc000 - 7f5b596dd000) ret: 0
[ 938.529482] nvidia-fs:vm_insert_page : 32 pages: ff919d80850f9d80 mapping: 0000000000000000, index: 164a79fe8020 (7f5b596dd000 - 7f5b596de000) ret: 0
[ 938.529483] nvidia-fs:vm_insert_page : 33 pages: ff919d80850f9dc0 mapping: 0000000000000000, index: 164a79fe8021 (7f5b596de000 - 7f5b596df000) ret: 0
[ 938.529485] nvidia-fs:vm_insert_page : 34 pages: ff919d80850f9e00 mapping: 0000000000000000, index: 164a79fe8022 (7f5b596df000 - 7f5b596e0000) ret: 0
[ 938.529486] nvidia-fs:vm_insert_page : 35 pages: ff919d80850f9e40 mapping: 0000000000000000, index: 164a79fe8023 (7f5b596e0000 - 7f5b596e1000) ret: 0
[ 938.529488] nvidia-fs:vm_insert_page : 36 pages: ff919d80850f9e80 mapping: 0000000000000000, index: 164a79fe8024 (7f5b596e1000 - 7f5b596e2000) ret: 0
[ 938.529491] nvidia-fs:vm_insert_page : 37 pages: ff919d80850f9ec0 mapping: 0000000000000000, index: 164a79fe8025 (7f5b596e2000 - 7f5b596e3000) ret: 0
[ 938.529493] nvidia-fs:vm_insert_page : 38 pages: ff919d80850fbc00 mapping: 0000000000000000, index: 164a79fe8026 (7f5b596e3000 - 7f5b596e4000) ret: 0
[ 938.529494] nvidia-fs:vm_insert_page : 39 pages: ff919d80850fbc40 mapping: 0000000000000000, index: 164a79fe8027 (7f5b596e4000 - 7f5b596e5000) ret: 0
[ 938.529495] nvidia-fs:vm_insert_page : 40 pages: ff919d80850fbc80 mapping: 0000000000000000, index: 164a79fe8028 (7f5b596e5000 - 7f5b596e6000) ret: 0
[ 938.529497] nvidia-fs:vm_insert_page : 41 pages: ff919d80850fbcc0 mapping: 0000000000000000, index: 164a79fe8029 (7f5b596e6000 - 7f5b596e7000) ret: 0
[ 938.529498] nvidia-fs:vm_insert_page : 42 pages: ff919d80850fbd00 mapping: 0000000000000000, index: 164a79fe802a (7f5b596e7000 - 7f5b596e8000) ret: 0
[ 938.529499] nvidia-fs:vm_insert_page : 43 pages: ff919d80850fbd40 mapping: 0000000000000000, index: 164a79fe802b (7f5b596e8000 - 7f5b596e9000) ret: 0
[ 938.529501] nvidia-fs:vm_insert_page : 44 pages: ff919d80850fbd80 mapping: 0000000000000000, index: 164a79fe802c (7f5b596e9000 - 7f5b596ea000) ret: 0
[ 938.529502] nvidia-fs:vm_insert_page : 45 pages: ff919d80850fbdc0 mapping: 0000000000000000, index: 164a79fe802d (7f5b596ea000 - 7f5b596eb000) ret: 0
[ 938.529504] nvidia-fs:vm_insert_page : 46 pages: ff919d80850fbe00 mapping: 0000000000000000, index: 164a79fe802e (7f5b596eb000 - 7f5b596ec000) ret: 0
[ 938.529505] nvidia-fs:vm_insert_page : 47 pages: ff919d80850fbe40 mapping: 0000000000000000, index: 164a79fe802f (7f5b596ec000 - 7f5b596ed000) ret: 0
[ 938.529507] nvidia-fs:vm_insert_page : 48 pages: ff919d80850fbe80 mapping: 0000000000000000, index: 164a79fe8030 (7f5b596ed000 - 7f5b596ee000) ret: 0
[ 938.529508] nvidia-fs:vm_insert_page : 49 pages: ff919d80850fbec0 mapping: 0000000000000000, index: 164a79fe8031 (7f5b596ee000 - 7f5b596ef000) ret: 0
[ 938.529509] nvidia-fs:vm_insert_page : 50 pages: ff919d80850fbf00 mapping: 0000000000000000, index: 164a79fe8032 (7f5b596ef000 - 7f5b596f0000) ret: 0
[ 938.529511] nvidia-fs:vm_insert_page : 51 pages: ff919d80850fbf40 mapping: 0000000000000000, index: 164a79fe8033 (7f5b596f0000 - 7f5b596f1000) ret: 0
[ 938.529512] nvidia-fs:vm_insert_page : 52 pages: ff919d80850fbf80 mapping: 0000000000000000, index: 164a79fe8034 (7f5b596f1000 - 7f5b596f2000) ret: 0
[ 938.529513] nvidia-fs:vm_insert_page : 53 pages: ff919d80850fbfc0 mapping: 0000000000000000, index: 164a79fe8035 (7f5b596f2000 - 7f5b596f3000) ret: 0
[ 938.529515] nvidia-fs:vm_insert_page : 54 pages: ff919d80850fc000 mapping: 0000000000000000, index: 164a79fe8036 (7f5b596f3000 - 7f5b596f4000) ret: 0
[ 938.529516] nvidia-fs:vm_insert_page : 55 pages: ff919d80850fc040 mapping: 0000000000000000, index: 164a79fe8037 (7f5b596f4000 - 7f5b596f5000) ret: 0
[ 938.529517] nvidia-fs:vm_insert_page : 56 pages: ff919d80850fc080 mapping: 0000000000000000, index: 164a79fe8038 (7f5b596f5000 - 7f5b596f6000) ret: 0
[ 938.529519] nvidia-fs:vm_insert_page : 57 pages: ff919d80850fc0c0 mapping: 0000000000000000, index: 164a79fe8039 (7f5b596f6000 - 7f5b596f7000) ret: 0
[ 938.529520] nvidia-fs:vm_insert_page : 58 pages: ff919d80850fc100 mapping: 0000000000000000, index: 164a79fe803a (7f5b596f7000 - 7f5b596f8000) ret: 0
[ 938.529522] nvidia-fs:vm_insert_page : 59 pages: ff919d80850fc140 mapping: 0000000000000000, index: 164a79fe803b (7f5b596f8000 - 7f5b596f9000) ret: 0
[ 938.529523] nvidia-fs:vm_insert_page : 60 pages: ff919d80850fc180 mapping: 0000000000000000, index: 164a79fe803c (7f5b596f9000 - 7f5b596fa000) ret: 0
[ 938.529524] nvidia-fs:vm_insert_page : 61 pages: ff919d80850fc1c0 mapping: 0000000000000000, index: 164a79fe803d (7f5b596fa000 - 7f5b596fb000) ret: 0
[ 938.529526] nvidia-fs:vm_insert_page : 62 pages: ff919d80850fc200 mapping: 0000000000000000, index: 164a79fe803e (7f5b596fb000 - 7f5b596fc000) ret: 0
[ 938.529527] nvidia-fs:vm_insert_page : 63 pages: ff919d80850fc240 mapping: 0000000000000000, index: 164a79fe803f (7f5b596fc000 - 7f5b596fd000) ret: 0
[ 938.529528] nvidia-fs:vm_insert_page : 64 pages: ff919d80850fc280 mapping: 0000000000000000, index: 164a79fe8040 (7f5b596fd000 - 7f5b596fe000) ret: 0
[ 938.529530] nvidia-fs:vm_insert_page : 65 pages: ff919d80850fc2c0 mapping: 0000000000000000, index: 164a79fe8041 (7f5b596fe000 - 7f5b596ff000) ret: 0
[ 938.529531] nvidia-fs:vm_insert_page : 66 pages: ff919d80850fc300 mapping: 0000000000000000, index: 164a79fe8042 (7f5b596ff000 - 7f5b59700000) ret: 0
[ 938.529533] nvidia-fs:vm_insert_page : 67 pages: ff919d80850fc340 mapping: 0000000000000000, index: 164a79fe8043 (7f5b59700000 - 7f5b59701000) ret: 0
[ 938.529534] nvidia-fs:vm_insert_page : 68 pages: ff919d80850fc380 mapping: 0000000000000000, index: 164a79fe8044 (7f5b59701000 - 7f5b59702000) ret: 0
[ 938.529535] nvidia-fs:vm_insert_page : 69 pages: ff919d80850fc3c0 mapping: 0000000000000000, index: 164a79fe8045 (7f5b59702000 - 7f5b59703000) ret: 0
[ 938.529537] nvidia-fs:vm_insert_page : 70 pages: ff919d80850fc400 mapping: 0000000000000000, index: 164a79fe8046 (7f5b59703000 - 7f5b59704000) ret: 0
[ 938.529538] nvidia-fs:vm_insert_page : 71 pages: ff919d80850fc440 mapping: 0000000000000000, index: 164a79fe8047 (7f5b59704000 - 7f5b59705000) ret: 0
[ 938.529539] nvidia-fs:vm_insert_page : 72 pages: ff919d80850fc480 mapping: 0000000000000000, index: 164a79fe8048 (7f5b59705000 - 7f5b59706000) ret: 0
[ 938.529541] nvidia-fs:vm_insert_page : 73 pages: ff919d80850fc4c0 mapping: 0000000000000000, index: 164a79fe8049 (7f5b59706000 - 7f5b59707000) ret: 0
[ 938.529542] nvidia-fs:vm_insert_page : 74 pages: ff919d80850fc500 mapping: 0000000000000000, index: 164a79fe804a (7f5b59707000 - 7f5b59708000) ret: 0
[ 938.529544] nvidia-fs:vm_insert_page : 75 pages: ff919d80850fc540 mapping: 0000000000000000, index: 164a79fe804b (7f5b59708000 - 7f5b59709000) ret: 0
[ 938.529545] nvidia-fs:vm_insert_page : 76 pages: ff919d80850fc580 mapping: 0000000000000000, index: 164a79fe804c (7f5b59709000 - 7f5b5970a000) ret: 0
[ 938.529547] nvidia-fs:vm_insert_page : 77 pages: ff919d80850fc5c0 mapping: 0000000000000000, index: 164a79fe804d (7f5b5970a000 - 7f5b5970b000) ret: 0
[ 938.529548] nvidia-fs:vm_insert_page : 78 pages: ff919d80850fc600 mapping: 0000000000000000, index: 164a79fe804e (7f5b5970b000 - 7f5b5970c000) ret: 0
[ 938.529549] nvidia-fs:vm_insert_page : 79 pages: ff919d80850fc640 mapping: 0000000000000000, index: 164a79fe804f (7f5b5970c000 - 7f5b5970d000) ret: 0
[ 938.529551] nvidia-fs:vm_insert_page : 80 pages: ff919d80850fc680 mapping: 0000000000000000, index: 164a79fe8050 (7f5b5970d000 - 7f5b5970e000) ret: 0
[ 938.529552] nvidia-fs:vm_insert_page : 81 pages: ff919d80850fc6c0 mapping: 0000000000000000, index: 164a79fe8051 (7f5b5970e000 - 7f5b5970f000) ret: 0
[ 938.529553] nvidia-fs:vm_insert_page : 82 pages: ff919d80850fc700 mapping: 0000000000000000, index: 164a79fe8052 (7f5b5970f000 - 7f5b59710000) ret: 0
[ 938.529555] nvidia-fs:vm_insert_page : 83 pages: ff919d80850fc740 mapping: 0000000000000000, index: 164a79fe8053 (7f5b59710000 - 7f5b59711000) ret: 0
[ 938.529556] nvidia-fs:vm_insert_page : 84 pages: ff919d80850fc780 mapping: 0000000000000000, index: 164a79fe8054 (7f5b59711000 - 7f5b59712000) ret: 0
[ 938.529557] nvidia-fs:vm_insert_page : 85 pages: ff919d80850fc7c0 mapping: 0000000000000000, index: 164a79fe8055 (7f5b59712000 - 7f5b59713000) ret: 0
[ 938.529559] nvidia-fs:vm_insert_page : 86 pages: ff919d80850fc800 mapping: 0000000000000000, index: 164a79fe8056 (7f5b59713000 - 7f5b59714000) ret: 0
[ 938.529560] nvidia-fs:vm_insert_page : 87 pages: ff919d80850fc840 mapping: 0000000000000000, index: 164a79fe8057 (7f5b59714000 - 7f5b59715000) ret: 0
[ 938.529561] nvidia-fs:vm_insert_page : 88 pages: ff919d80850fc880 mapping: 0000000000000000, index: 164a79fe8058 (7f5b59715000 - 7f5b59716000) ret: 0
[ 938.529563] nvidia-fs:vm_insert_page : 89 pages: ff919d80850fc8c0 mapping: 0000000000000000, index: 164a79fe8059 (7f5b59716000 - 7f5b59717000) ret: 0
[ 938.529564] nvidia-fs:vm_insert_page : 90 pages: ff919d80850fc900 mapping: 0000000000000000, index: 164a79fe805a (7f5b59717000 - 7f5b59718000) ret: 0
[ 938.529565] nvidia-fs:vm_insert_page : 91 pages: ff919d80850fc940 mapping: 0000000000000000, index: 164a79fe805b (7f5b59718000 - 7f5b59719000) ret: 0
[ 938.529567] nvidia-fs:vm_insert_page : 92 pages: ff919d80850fc980 mapping: 0000000000000000, index: 164a79fe805c (7f5b59719000 - 7f5b5971a000) ret: 0
[ 938.529568] nvidia-fs:vm_insert_page : 93 pages: ff919d80850fc9c0 mapping: 0000000000000000, index: 164a79fe805d (7f5b5971a000 - 7f5b5971b000) ret: 0
[ 938.529570] nvidia-fs:vm_insert_page : 94 pages: ff919d80850fca00 mapping: 0000000000000000, index: 164a79fe805e (7f5b5971b000 - 7f5b5971c000) ret: 0
[ 938.529571] nvidia-fs:vm_insert_page : 95 pages: ff919d80850fca40 mapping: 0000000000000000, index: 164a79fe805f (7f5b5971c000 - 7f5b5971d000) ret: 0
[ 938.529572] nvidia-fs:vm_insert_page : 96 pages: ff919d80850fca80 mapping: 0000000000000000, index: 164a79fe8060 (7f5b5971d000 - 7f5b5971e000) ret: 0
[ 938.529574] nvidia-fs:vm_insert_page : 97 pages: ff919d80850fcac0 mapping: 0000000000000000, index: 164a79fe8061 (7f5b5971e000 - 7f5b5971f000) ret: 0
[ 938.529575] nvidia-fs:vm_insert_page : 98 pages: ff919d80850fcb00 mapping: 0000000000000000, index: 164a79fe8062 (7f5b5971f000 - 7f5b59720000) ret: 0
[ 938.529577] nvidia-fs:vm_insert_page : 99 pages: ff919d80850fcb40 mapping: 0000000000000000, index: 164a79fe8063 (7f5b59720000 - 7f5b59721000) ret: 0
[ 938.529579] nvidia-fs:vm_insert_page : 100 pages: ff919d80850fcb80 mapping: 0000000000000000, index: 164a79fe8064 (7f5b59721000 - 7f5b59722000) ret: 0
[ 938.529581] nvidia-fs:vm_insert_page : 101 pages: ff919d80850fcbc0 mapping: 0000000000000000, index: 164a79fe8065 (7f5b59722000 - 7f5b59723000) ret: 0
[ 938.529582] nvidia-fs:vm_insert_page : 102 pages: ff919d80850fcc00 mapping: 0000000000000000, index: 164a79fe8066 (7f5b59723000 - 7f5b59724000) ret: 0
[ 938.529584] nvidia-fs:vm_insert_page : 103 pages: ff919d80850fcc40 mapping: 0000000000000000, index: 164a79fe8067 (7f5b59724000 - 7f5b59725000) ret: 0
[ 938.529585] nvidia-fs:vm_insert_page : 104 pages: ff919d80850fcc80 mapping: 0000000000000000, index: 164a79fe8068 (7f5b59725000 - 7f5b59726000) ret: 0
[ 938.529586] nvidia-fs:vm_insert_page : 105 pages: ff919d80850fccc0 mapping: 0000000000000000, index: 164a79fe8069 (7f5b59726000 - 7f5b59727000) ret: 0
[ 938.529588] nvidia-fs:vm_insert_page : 106 pages: ff919d80850fcd00 mapping: 0000000000000000, index: 164a79fe806a (7f5b59727000 - 7f5b59728000) ret: 0
[ 938.529589] nvidia-fs:vm_insert_page : 107 pages: ff919d80850fcd40 mapping: 0000000000000000, index: 164a79fe806b (7f5b59728000 - 7f5b59729000) ret: 0
[ 938.529591] nvidia-fs:vm_insert_page : 108 pages: ff919d80850fcd80 mapping: 0000000000000000, index: 164a79fe806c (7f5b59729000 - 7f5b5972a000) ret: 0
[ 938.529592] nvidia-fs:vm_insert_page : 109 pages: ff919d80850fcdc0 mapping: 0000000000000000, index: 164a79fe806d (7f5b5972a000 - 7f5b5972b000) ret: 0
[ 938.529593] nvidia-fs:vm_insert_page : 110 pages: ff919d80850fce00 mapping: 0000000000000000, index: 164a79fe806e (7f5b5972b000 - 7f5b5972c000) ret: 0
[ 938.529595] nvidia-fs:vm_insert_page : 111 pages: ff919d80850fce40 mapping: 0000000000000000, index: 164a79fe806f (7f5b5972c000 - 7f5b5972d000) ret: 0
[ 938.529596] nvidia-fs:vm_insert_page : 112 pages: ff919d80850fce80 mapping: 0000000000000000, index: 164a79fe8070 (7f5b5972d000 - 7f5b5972e000) ret: 0
[ 938.529597] nvidia-fs:vm_insert_page : 113 pages: ff919d80850fcec0 mapping: 0000000000000000, index: 164a79fe8071 (7f5b5972e000 - 7f5b5972f000) ret: 0
[ 938.529599] nvidia-fs:vm_insert_page : 114 pages: ff919d80850fcf00 mapping: 0000000000000000, index: 164a79fe8072 (7f5b5972f000 - 7f5b59730000) ret: 0
[ 938.529600] nvidia-fs:vm_insert_page : 115 pages: ff919d80850fcf40 mapping: 0000000000000000, index: 164a79fe8073 (7f5b59730000 - 7f5b59731000) ret: 0
[ 938.529602] nvidia-fs:vm_insert_page : 116 pages: ff919d80850fcf80 mapping: 0000000000000000, index: 164a79fe8074 (7f5b59731000 - 7f5b59732000) ret: 0
[ 938.529603] nvidia-fs:vm_insert_page : 117 pages: ff919d80850fcfc0 mapping: 0000000000000000, index: 164a79fe8075 (7f5b59732000 - 7f5b59733000) ret: 0
[ 938.529604] nvidia-fs:vm_insert_page : 118 pages: ff919d80850fd000 mapping: 0000000000000000, index: 164a79fe8076 (7f5b59733000 - 7f5b59734000) ret: 0
[ 938.529606] nvidia-fs:vm_insert_page : 119 pages: ff919d80850fd040 mapping: 0000000000000000, index: 164a79fe8077 (7f5b59734000 - 7f5b59735000) ret: 0
[ 938.529607] nvidia-fs:vm_insert_page : 120 pages: ff919d80850fd080 mapping: 0000000000000000, index: 164a79fe8078 (7f5b59735000 - 7f5b59736000) ret: 0
[ 938.529608] nvidia-fs:vm_insert_page : 121 pages: ff919d80850fd0c0 mapping: 0000000000000000, index: 164a79fe8079 (7f5b59736000 - 7f5b59737000) ret: 0
[ 938.529610] nvidia-fs:vm_insert_page : 122 pages: ff919d80850fd100 mapping: 0000000000000000, index: 164a79fe807a (7f5b59737000 - 7f5b59738000) ret: 0
[ 938.529611] nvidia-fs:vm_insert_page : 123 pages: ff919d80850fd140 mapping: 0000000000000000, index: 164a79fe807b (7f5b59738000 - 7f5b59739000) ret: 0
[ 938.529612] nvidia-fs:vm_insert_page : 124 pages: ff919d80850fd180 mapping: 0000000000000000, index: 164a79fe807c (7f5b59739000 - 7f5b5973a000) ret: 0
[ 938.529614] nvidia-fs:vm_insert_page : 125 pages: ff919d80850fd1c0 mapping: 0000000000000000, index: 164a79fe807d (7f5b5973a000 - 7f5b5973b000) ret: 0
[ 938.529615] nvidia-fs:vm_insert_page : 126 pages: ff919d80850fd200 mapping: 0000000000000000, index: 164a79fe807e (7f5b5973b000 - 7f5b5973c000) ret: 0
[ 938.529617] nvidia-fs:vm_insert_page : 127 pages: ff919d80850fd240 mapping: 0000000000000000, index: 164a79fe807f (7f5b5973c000 - 7f5b5973d000) ret: 0
[ 938.529618] nvidia-fs:vm_insert_page : 128 pages: ff919d80850fd280 mapping: 0000000000000000, index: 164a79fe8080 (7f5b5973d000 - 7f5b5973e000) ret: 0
[ 938.529620] nvidia-fs:vm_insert_page : 129 pages: ff919d80850fd2c0 mapping: 0000000000000000, index: 164a79fe8081 (7f5b5973e000 - 7f5b5973f000) ret: 0
[ 938.529621] nvidia-fs:vm_insert_page : 130 pages: ff919d80850fd300 mapping: 0000000000000000, index: 164a79fe8082 (7f5b5973f000 - 7f5b59740000) ret: 0
[ 938.529623] nvidia-fs:vm_insert_page : 131 pages: ff919d80850fd340 mapping: 0000000000000000, index: 164a79fe8083 (7f5b59740000 - 7f5b59741000) ret: 0
[ 938.529625] nvidia-fs:vm_insert_page : 132 pages: ff919d80850fd380 mapping: 0000000000000000, index: 164a79fe8084 (7f5b59741000 - 7f5b59742000) ret: 0
[ 938.529627] nvidia-fs:vm_insert_page : 133 pages: ff919d80850fd3c0 mapping: 0000000000000000, index: 164a79fe8085 (7f5b59742000 - 7f5b59743000) ret: 0
[ 938.529628] nvidia-fs:vm_insert_page : 134 pages: ff919d80850fd400 mapping: 0000000000000000, index: 164a79fe8086 (7f5b59743000 - 7f5b59744000) ret: 0
[ 938.529630] nvidia-fs:vm_insert_page : 135 pages: ff919d80850fd440 mapping: 0000000000000000, index: 164a79fe8087 (7f5b59744000 - 7f5b59745000) ret: 0
[ 938.529632] nvidia-fs:vm_insert_page : 136 pages: ff919d80850fd480 mapping: 0000000000000000, index: 164a79fe8088 (7f5b59745000 - 7f5b59746000) ret: 0
[ 938.529633] nvidia-fs:vm_insert_page : 137 pages: ff919d80850fd4c0 mapping: 0000000000000000, index: 164a79fe8089 (7f5b59746000 - 7f5b59747000) ret: 0
[ 938.529635] nvidia-fs:vm_insert_page : 138 pages: ff919d80850fd500 mapping: 0000000000000000, index: 164a79fe808a (7f5b59747000 - 7f5b59748000) ret: 0
[ 938.529637] nvidia-fs:vm_insert_page : 139 pages: ff919d80850fd540 mapping: 0000000000000000, index: 164a79fe808b (7f5b59748000 - 7f5b59749000) ret: 0
[ 938.529638] nvidia-fs:vm_insert_page : 140 pages: ff919d80850fd580 mapping: 0000000000000000, index: 164a79fe808c (7f5b59749000 - 7f5b5974a000) ret: 0
[ 938.529640] nvidia-fs:vm_insert_page : 141 pages: ff919d80850fd5c0 mapping: 0000000000000000, index: 164a79fe808d (7f5b5974a000 - 7f5b5974b000) ret: 0
[ 938.529642] nvidia-fs:vm_insert_page : 142 pages: ff919d80850fd600 mapping: 0000000000000000, index: 164a79fe808e (7f5b5974b000 - 7f5b5974c000) ret: 0
[ 938.529643] nvidia-fs:vm_insert_page : 143 pages: ff919d80850fd640 mapping: 0000000000000000, index: 164a79fe808f (7f5b5974c000 - 7f5b5974d000) ret: 0
[ 938.529645] nvidia-fs:vm_insert_page : 144 pages: ff919d80850fd680 mapping: 0000000000000000, index: 164a79fe8090 (7f5b5974d000 - 7f5b5974e000) ret: 0
[ 938.529647] nvidia-fs:vm_insert_page : 145 pages: ff919d80850fd6c0 mapping: 0000000000000000, index: 164a79fe8091 (7f5b5974e000 - 7f5b5974f000) ret: 0
[ 938.529649] nvidia-fs:vm_insert_page : 146 pages: ff919d80850fd700 mapping: 0000000000000000, index: 164a79fe8092 (7f5b5974f000 - 7f5b59750000) ret: 0
[ 938.529650] nvidia-fs:vm_insert_page : 147 pages: ff919d80850fd740 mapping: 0000000000000000, index: 164a79fe8093 (7f5b59750000 - 7f5b59751000) ret: 0
[ 938.529652] nvidia-fs:vm_insert_page : 148 pages: ff919d80850fd780 mapping: 0000000000000000, index: 164a79fe8094 (7f5b59751000 - 7f5b59752000) ret: 0
[ 938.529654] nvidia-fs:vm_insert_page : 149 pages: ff919d80850fd7c0 mapping: 0000000000000000, index: 164a79fe8095 (7f5b59752000 - 7f5b59753000) ret: 0
[ 938.529666] nvidia-fs:vm_insert_page : 150 pages: ff919d80850fd800 mapping: 0000000000000000, index: 164a79fe8096 (7f5b59753000 - 7f5b59754000) ret: 0
[ 938.529669] nvidia-fs:vm_insert_page : 151 pages: ff919d80850fd840 mapping: 0000000000000000, index: 164a79fe8097 (7f5b59754000 - 7f5b59755000) ret: 0
[ 938.529672] nvidia-fs:vm_insert_page : 152 pages: ff919d80850fd880 mapping: 0000000000000000, index: 164a79fe8098 (7f5b59755000 - 7f5b59756000) ret: 0
[ 938.529674] nvidia-fs:vm_insert_page : 153 pages: ff919d80850fd8c0 mapping: 0000000000000000, index: 164a79fe8099 (7f5b59756000 - 7f5b59757000) ret: 0
[ 938.529676] nvidia-fs:vm_insert_page : 154 pages: ff919d80850fd900 mapping: 0000000000000000, index: 164a79fe809a (7f5b59757000 - 7f5b59758000) ret: 0
[ 938.529678] nvidia-fs:vm_insert_page : 155 pages: ff919d80850fd940 mapping: 0000000000000000, index: 164a79fe809b (7f5b59758000 - 7f5b59759000) ret: 0
[ 938.529679] nvidia-fs:vm_insert_page : 156 pages: ff919d80850fd980 mapping: 0000000000000000, index: 164a79fe809c (7f5b59759000 - 7f5b5975a000) ret: 0
[ 938.529681] nvidia-fs:vm_insert_page : 157 pages: ff919d80850fd9c0 mapping: 0000000000000000, index: 164a79fe809d (7f5b5975a000 - 7f5b5975b000) ret: 0
[ 938.529683] nvidia-fs:vm_insert_page : 158 pages: ff919d80850fda00 mapping: 0000000000000000, index: 164a79fe809e (7f5b5975b000 - 7f5b5975c000) ret: 0
[ 938.529685] nvidia-fs:vm_insert_page : 159 pages: ff919d80850fda40 mapping: 0000000000000000, index: 164a79fe809f (7f5b5975c000 - 7f5b5975d000) ret: 0
[ 938.529686] nvidia-fs:vm_insert_page : 160 pages: ff919d80850fda80 mapping: 0000000000000000, index: 164a79fe80a0 (7f5b5975d000 - 7f5b5975e000) ret: 0
[ 938.529688] nvidia-fs:vm_insert_page : 161 pages: ff919d80850fdac0 mapping: 0000000000000000, index: 164a79fe80a1 (7f5b5975e000 - 7f5b5975f000) ret: 0
[ 938.529689] nvidia-fs:vm_insert_page : 162 pages: ff919d80850fdb00 mapping: 0000000000000000, index: 164a79fe80a2 (7f5b5975f000 - 7f5b59760000) ret: 0
[ 938.529692] nvidia-fs:vm_insert_page : 163 pages: ff919d80850fdb40 mapping: 0000000000000000, index: 164a79fe80a3 (7f5b59760000 - 7f5b59761000) ret: 0
[ 938.529694] nvidia-fs:vm_insert_page : 164 pages: ff919d80850fdb80 mapping: 0000000000000000, index: 164a79fe80a4 (7f5b59761000 - 7f5b59762000) ret: 0
[ 938.529695] nvidia-fs:vm_insert_page : 165 pages: ff919d80850fdbc0 mapping: 0000000000000000, index: 164a79fe80a5 (7f5b59762000 - 7f5b59763000) ret: 0
[ 938.529697] nvidia-fs:vm_insert_page : 166 pages: ff919d80850fdc00 mapping: 0000000000000000, index: 164a79fe80a6 (7f5b59763000 - 7f5b59764000) ret: 0
[ 938.529698] nvidia-fs:vm_insert_page : 167 pages: ff919d80850fdc40 mapping: 0000000000000000, index: 164a79fe80a7 (7f5b59764000 - 7f5b59765000) ret: 0
[ 938.529700] nvidia-fs:vm_insert_page : 168 pages: ff919d80850fdc80 mapping: 0000000000000000, index: 164a79fe80a8 (7f5b59765000 - 7f5b59766000) ret: 0
[ 938.529702] nvidia-fs:vm_insert_page : 169 pages: ff919d80850fdcc0 mapping: 0000000000000000, index: 164a79fe80a9 (7f5b59766000 - 7f5b59767000) ret: 0
[ 938.529704] nvidia-fs:vm_insert_page : 170 pages: ff919d80850fdd00 mapping: 0000000000000000, index: 164a79fe80aa (7f5b59767000 - 7f5b59768000) ret: 0
[ 938.529705] nvidia-fs:vm_insert_page : 171 pages: ff919d80850fdd40 mapping: 0000000000000000, index: 164a79fe80ab (7f5b59768000 - 7f5b59769000) ret: 0
[ 938.529706] nvidia-fs:vm_insert_page : 172 pages: ff919d80850fdd80 mapping: 0000000000000000, index: 164a79fe80ac (7f5b59769000 - 7f5b5976a000) ret: 0
[ 938.529708] nvidia-fs:vm_insert_page : 173 pages: ff919d80850fddc0 mapping: 0000000000000000, index: 164a79fe80ad (7f5b5976a000 - 7f5b5976b000) ret: 0
[ 938.529709] nvidia-fs:vm_insert_page : 174 pages: ff919d80850fde00 mapping: 0000000000000000, index: 164a79fe80ae (7f5b5976b000 - 7f5b5976c000) ret: 0
[ 938.529711] nvidia-fs:vm_insert_page : 175 pages: ff919d80850fde40 mapping: 0000000000000000, index: 164a79fe80af (7f5b5976c000 - 7f5b5976d000) ret: 0
[ 938.529713] nvidia-fs:vm_insert_page : 176 pages: ff919d80850fde80 mapping: 0000000000000000, index: 164a79fe80b0 (7f5b5976d000 - 7f5b5976e000) ret: 0
[ 938.529714] nvidia-fs:vm_insert_page : 177 pages: ff919d80850fdec0 mapping: 0000000000000000, index: 164a79fe80b1 (7f5b5976e000 - 7f5b5976f000) ret: 0
[ 938.529716] nvidia-fs:vm_insert_page : 178 pages: ff919d80850fdf00 mapping: 0000000000000000, index: 164a79fe80b2 (7f5b5976f000 - 7f5b59770000) ret: 0
[ 938.529717] nvidia-fs:vm_insert_page : 179 pages: ff919d80850fdf40 mapping: 0000000000000000, index: 164a79fe80b3 (7f5b59770000 - 7f5b59771000) ret: 0
[ 938.529719] nvidia-fs:vm_insert_page : 180 pages: ff919d80850fdf80 mapping: 0000000000000000, index: 164a79fe80b4 (7f5b59771000 - 7f5b59772000) ret: 0
[ 938.529720] nvidia-fs:vm_insert_page : 181 pages: ff919d80850fdfc0 mapping: 0000000000000000, index: 164a79fe80b5 (7f5b59772000 - 7f5b59773000) ret: 0
[ 938.529722] nvidia-fs:vm_insert_page : 182 pages: ff919d80850fe000 mapping: 0000000000000000, index: 164a79fe80b6 (7f5b59773000 - 7f5b59774000) ret: 0
[ 938.529724] nvidia-fs:vm_insert_page : 183 pages: ff919d80850fe040 mapping: 0000000000000000, index: 164a79fe80b7 (7f5b59774000 - 7f5b59775000) ret: 0
[ 938.529726] nvidia-fs:vm_insert_page : 184 pages: ff919d80850fe080 mapping: 0000000000000000, index: 164a79fe80b8 (7f5b59775000 - 7f5b59776000) ret: 0
[ 938.529728] nvidia-fs:vm_insert_page : 185 pages: ff919d80850fe0c0 mapping: 0000000000000000, index: 164a79fe80b9 (7f5b59776000 - 7f5b59777000) ret: 0
[ 938.529730] nvidia-fs:vm_insert_page : 186 pages: ff919d80850fe100 mapping: 0000000000000000, index: 164a79fe80ba (7f5b59777000 - 7f5b59778000) ret: 0
[ 938.529732] nvidia-fs:vm_insert_page : 187 pages: ff919d80850fe140 mapping: 0000000000000000, index: 164a79fe80bb (7f5b59778000 - 7f5b59779000) ret: 0
[ 938.529734] nvidia-fs:vm_insert_page : 188 pages: ff919d80850fe180 mapping: 0000000000000000, index: 164a79fe80bc (7f5b59779000 - 7f5b5977a000) ret: 0
[ 938.529735] nvidia-fs:vm_insert_page : 189 pages: ff919d80850fe1c0 mapping: 0000000000000000, index: 164a79fe80bd (7f5b5977a000 - 7f5b5977b000) ret: 0
[ 938.529736] nvidia-fs:vm_insert_page : 190 pages: ff919d80850fe200 mapping: 0000000000000000, index: 164a79fe80be (7f5b5977b000 - 7f5b5977c000) ret: 0
[ 938.529738] nvidia-fs:vm_insert_page : 191 pages: ff919d80850fe240 mapping: 0000000000000000, index: 164a79fe80bf (7f5b5977c000 - 7f5b5977d000) ret: 0
[ 938.529739] nvidia-fs:vm_insert_page : 192 pages: ff919d80850fe280 mapping: 0000000000000000, index: 164a79fe80c0 (7f5b5977d000 - 7f5b5977e000) ret: 0
[ 938.529741] nvidia-fs:vm_insert_page : 193 pages: ff919d80850fe2c0 mapping: 0000000000000000, index: 164a79fe80c1 (7f5b5977e000 - 7f5b5977f000) ret: 0
[ 938.529743] nvidia-fs:vm_insert_page : 194 pages: ff919d80850fe300 mapping: 0000000000000000, index: 164a79fe80c2 (7f5b5977f000 - 7f5b59780000) ret: 0
[ 938.529744] nvidia-fs:vm_insert_page : 195 pages: ff919d80850fe340 mapping: 0000000000000000, index: 164a79fe80c3 (7f5b59780000 - 7f5b59781000) ret: 0
[ 938.529746] nvidia-fs:vm_insert_page : 196 pages: ff919d80850fe380 mapping: 0000000000000000, index: 164a79fe80c4 (7f5b59781000 - 7f5b59782000) ret: 0
[ 938.529747] nvidia-fs:vm_insert_page : 197 pages: ff919d80850fe3c0 mapping: 0000000000000000, index: 164a79fe80c5 (7f5b59782000 - 7f5b59783000) ret: 0
[ 938.529748] nvidia-fs:vm_insert_page : 198 pages: ff919d80850fe400 mapping: 0000000000000000, index: 164a79fe80c6 (7f5b59783000 - 7f5b59784000) ret: 0
[ 938.529751] nvidia-fs:vm_insert_page : 199 pages: ff919d80850fe440 mapping: 0000000000000000, index: 164a79fe80c7 (7f5b59784000 - 7f5b59785000) ret: 0
[ 938.529753] nvidia-fs:vm_insert_page : 200 pages: ff919d80850fe480 mapping: 0000000000000000, index: 164a79fe80c8 (7f5b59785000 - 7f5b59786000) ret: 0
[ 938.529755] nvidia-fs:vm_insert_page : 201 pages: ff919d80850fe4c0 mapping: 0000000000000000, index: 164a79fe80c9 (7f5b59786000 - 7f5b59787000) ret: 0
[ 938.529756] nvidia-fs:vm_insert_page : 202 pages: ff919d80850fe500 mapping: 0000000000000000, index: 164a79fe80ca (7f5b59787000 - 7f5b59788000) ret: 0
[ 938.529757] nvidia-fs:vm_insert_page : 203 pages: ff919d80850fe540 mapping: 0000000000000000, index: 164a79fe80cb (7f5b59788000 - 7f5b59789000) ret: 0
[ 938.529760] nvidia-fs:vm_insert_page : 204 pages: ff919d80850fe580 mapping: 0000000000000000, index: 164a79fe80cc (7f5b59789000 - 7f5b5978a000) ret: 0
[ 938.529762] nvidia-fs:vm_insert_page : 205 pages: ff919d80850fe5c0 mapping: 0000000000000000, index: 164a79fe80cd (7f5b5978a000 - 7f5b5978b000) ret: 0
[ 938.529763] nvidia-fs:vm_insert_page : 206 pages: ff919d80850fe600 mapping: 0000000000000000, index: 164a79fe80ce (7f5b5978b000 - 7f5b5978c000) ret: 0
[ 938.529765] nvidia-fs:vm_insert_page : 207 pages: ff919d80850fe640 mapping: 0000000000000000, index: 164a79fe80cf (7f5b5978c000 - 7f5b5978d000) ret: 0
[ 938.529766] nvidia-fs:vm_insert_page : 208 pages: ff919d80850fe680 mapping: 0000000000000000, index: 164a79fe80d0 (7f5b5978d000 - 7f5b5978e000) ret: 0
[ 938.529767] nvidia-fs:vm_insert_page : 209 pages: ff919d80850fe6c0 mapping: 0000000000000000, index: 164a79fe80d1 (7f5b5978e000 - 7f5b5978f000) ret: 0
[ 938.529769] nvidia-fs:vm_insert_page : 210 pages: ff919d80850fe700 mapping: 0000000000000000, index: 164a79fe80d2 (7f5b5978f000 - 7f5b59790000) ret: 0
[ 938.529771] nvidia-fs:vm_insert_page : 211 pages: ff919d80850fe740 mapping: 0000000000000000, index: 164a79fe80d3 (7f5b59790000 - 7f5b59791000) ret: 0
[ 938.529772] nvidia-fs:vm_insert_page : 212 pages: ff919d80850fe780 mapping: 0000000000000000, index: 164a79fe80d4 (7f5b59791000 - 7f5b59792000) ret: 0
[ 938.529774] nvidia-fs:vm_insert_page : 213 pages: ff919d80850fe7c0 mapping: 0000000000000000, index: 164a79fe80d5 (7f5b59792000 - 7f5b59793000) ret: 0
[ 938.529775] nvidia-fs:vm_insert_page : 214 pages: ff919d80850fe800 mapping: 0000000000000000, index: 164a79fe80d6 (7f5b59793000 - 7f5b59794000) ret: 0
[ 938.529776] nvidia-fs:vm_insert_page : 215 pages: ff919d80850fe840 mapping: 0000000000000000, index: 164a79fe80d7 (7f5b59794000 - 7f5b59795000) ret: 0
[ 938.529778] nvidia-fs:vm_insert_page : 216 pages: ff919d80850fe880 mapping: 0000000000000000, index: 164a79fe80d8 (7f5b59795000 - 7f5b59796000) ret: 0
[ 938.529779] nvidia-fs:vm_insert_page : 217 pages: ff919d80850fe8c0 mapping: 0000000000000000, index: 164a79fe80d9 (7f5b59796000 - 7f5b59797000) ret: 0
[ 938.529781] nvidia-fs:vm_insert_page : 218 pages: ff919d80850fe900 mapping: 0000000000000000, index: 164a79fe80da (7f5b59797000 - 7f5b59798000) ret: 0
[ 938.529782] nvidia-fs:vm_insert_page : 219 pages: ff919d80850fe940 mapping: 0000000000000000, index: 164a79fe80db (7f5b59798000 - 7f5b59799000) ret: 0
[ 938.529784] nvidia-fs:vm_insert_page : 220 pages: ff919d80850fe980 mapping: 0000000000000000, index: 164a79fe80dc (7f5b59799000 - 7f5b5979a000) ret: 0
[ 938.529785] nvidia-fs:vm_insert_page : 221 pages: ff919d80850fe9c0 mapping: 0000000000000000, index: 164a79fe80dd (7f5b5979a000 - 7f5b5979b000) ret: 0
[ 938.529786] nvidia-fs:vm_insert_page : 222 pages: ff919d80850fea00 mapping: 0000000000000000, index: 164a79fe80de (7f5b5979b000 - 7f5b5979c000) ret: 0
[ 938.529789] nvidia-fs:vm_insert_page : 223 pages: ff919d80850fea40 mapping: 0000000000000000, index: 164a79fe80df (7f5b5979c000 - 7f5b5979d000) ret: 0
[ 938.529791] nvidia-fs:vm_insert_page : 224 pages: ff919d80850fea80 mapping: 0000000000000000, index: 164a79fe80e0 (7f5b5979d000 - 7f5b5979e000) ret: 0
[ 938.529792] nvidia-fs:vm_insert_page : 225 pages: ff919d80850feac0 mapping: 0000000000000000, index: 164a79fe80e1 (7f5b5979e000 - 7f5b5979f000) ret: 0
[ 938.529795] nvidia-fs:vm_insert_page : 226 pages: ff919d80850feb00 mapping: 0000000000000000, index: 164a79fe80e2 (7f5b5979f000 - 7f5b597a0000) ret: 0
[ 938.529796] nvidia-fs:vm_insert_page : 227 pages: ff919d80850feb40 mapping: 0000000000000000, index: 164a79fe80e3 (7f5b597a0000 - 7f5b597a1000) ret: 0
[ 938.529798] nvidia-fs:vm_insert_page : 228 pages: ff919d80850feb80 mapping: 0000000000000000, index: 164a79fe80e4 (7f5b597a1000 - 7f5b597a2000) ret: 0
[ 938.529800] nvidia-fs:vm_insert_page : 229 pages: ff919d80850febc0 mapping: 0000000000000000, index: 164a79fe80e5 (7f5b597a2000 - 7f5b597a3000) ret: 0
[ 938.529801] nvidia-fs:vm_insert_page : 230 pages: ff919d80850fec00 mapping: 0000000000000000, index: 164a79fe80e6 (7f5b597a3000 - 7f5b597a4000) ret: 0
[ 938.529802] nvidia-fs:vm_insert_page : 231 pages: ff919d80850fec40 mapping: 0000000000000000, index: 164a79fe80e7 (7f5b597a4000 - 7f5b597a5000) ret: 0
[ 938.529804] nvidia-fs:vm_insert_page : 232 pages: ff919d80850fec80 mapping: 0000000000000000, index: 164a79fe80e8 (7f5b597a5000 - 7f5b597a6000) ret: 0
[ 938.529805] nvidia-fs:vm_insert_page : 233 pages: ff919d80850fecc0 mapping: 0000000000000000, index: 164a79fe80e9 (7f5b597a6000 - 7f5b597a7000) ret: 0
[ 938.529807] nvidia-fs:vm_insert_page : 234 pages: ff919d80850fed00 mapping: 0000000000000000, index: 164a79fe80ea (7f5b597a7000 - 7f5b597a8000) ret: 0
[ 938.529808] nvidia-fs:vm_insert_page : 235 pages: ff919d80850fed40 mapping: 0000000000000000, index: 164a79fe80eb (7f5b597a8000 - 7f5b597a9000) ret: 0
[ 938.529810] nvidia-fs:vm_insert_page : 236 pages: ff919d80850fed80 mapping: 0000000000000000, index: 164a79fe80ec (7f5b597a9000 - 7f5b597aa000) ret: 0
[ 938.529811] nvidia-fs:vm_insert_page : 237 pages: ff919d80850fedc0 mapping: 0000000000000000, index: 164a79fe80ed (7f5b597aa000 - 7f5b597ab000) ret: 0
[ 938.529812] nvidia-fs:vm_insert_page : 238 pages: ff919d80850fee00 mapping: 0000000000000000, index: 164a79fe80ee (7f5b597ab000 - 7f5b597ac000) ret: 0
[ 938.529814] nvidia-fs:vm_insert_page : 239 pages: ff919d80850fee40 mapping: 0000000000000000, index: 164a79fe80ef (7f5b597ac000 - 7f5b597ad000) ret: 0
[ 938.529816] nvidia-fs:vm_insert_page : 240 pages: ff919d80850fee80 mapping: 0000000000000000, index: 164a79fe80f0 (7f5b597ad000 - 7f5b597ae000) ret: 0
[ 938.529817] nvidia-fs:vm_insert_page : 241 pages: ff919d80850feec0 mapping: 0000000000000000, index: 164a79fe80f1 (7f5b597ae000 - 7f5b597af000) ret: 0
[ 938.529818] nvidia-fs:vm_insert_page : 242 pages: ff919d80850fef00 mapping: 0000000000000000, index: 164a79fe80f2 (7f5b597af000 - 7f5b597b0000) ret: 0
[ 938.529820] nvidia-fs:vm_insert_page : 243 pages: ff919d80850fef40 mapping: 0000000000000000, index: 164a79fe80f3 (7f5b597b0000 - 7f5b597b1000) ret: 0
[ 938.529821] nvidia-fs:vm_insert_page : 244 pages: ff919d80850fef80 mapping: 0000000000000000, index: 164a79fe80f4 (7f5b597b1000 - 7f5b597b2000) ret: 0
[ 938.529823] nvidia-fs:vm_insert_page : 245 pages: ff919d80850fefc0 mapping: 0000000000000000, index: 164a79fe80f5 (7f5b597b2000 - 7f5b597b3000) ret: 0
[ 938.529824] nvidia-fs:vm_insert_page : 246 pages: ff919d80850ff000 mapping: 0000000000000000, index: 164a79fe80f6 (7f5b597b3000 - 7f5b597b4000) ret: 0
[ 938.529826] nvidia-fs:vm_insert_page : 247 pages: ff919d80850ff040 mapping: 0000000000000000, index: 164a79fe80f7 (7f5b597b4000 - 7f5b597b5000) ret: 0
[ 938.529827] nvidia-fs:vm_insert_page : 248 pages: ff919d80850ff080 mapping: 0000000000000000, index: 164a79fe80f8 (7f5b597b5000 - 7f5b597b6000) ret: 0
[ 938.529829] nvidia-fs:vm_insert_page : 249 pages: ff919d80850ff0c0 mapping: 0000000000000000, index: 164a79fe80f9 (7f5b597b6000 - 7f5b597b7000) ret: 0
[ 938.529830] nvidia-fs:vm_insert_page : 250 pages: ff919d80850ff100 mapping: 0000000000000000, index: 164a79fe80fa (7f5b597b7000 - 7f5b597b8000) ret: 0
[ 938.529832] nvidia-fs:vm_insert_page : 251 pages: ff919d80850ff140 mapping: 0000000000000000, index: 164a79fe80fb (7f5b597b8000 - 7f5b597b9000) ret: 0
[ 938.529833] nvidia-fs:vm_insert_page : 252 pages: ff919d80850ff180 mapping: 0000000000000000, index: 164a79fe80fc (7f5b597b9000 - 7f5b597ba000) ret: 0
[ 938.529835] nvidia-fs:vm_insert_page : 253 pages: ff919d80850ff1c0 mapping: 0000000000000000, index: 164a79fe80fd (7f5b597ba000 - 7f5b597bb000) ret: 0
[ 938.529836] nvidia-fs:vm_insert_page : 254 pages: ff919d80850ff200 mapping: 0000000000000000, index: 164a79fe80fe (7f5b597bb000 - 7f5b597bc000) ret: 0
[ 938.529837] nvidia-fs:vm_insert_page : 255 pages: ff919d80850ff240 mapping: 0000000000000000, index: 164a79fe80ff (7f5b597bc000 - 7f5b597bd000) ret: 0
[ 938.529839] nvidia-fs:page ff453fd243e94800 mmap (7f5b596bd000 - 7f5b597bd000), len:1048576 success vma:00000000dd88f1d2, file:0000000030458933 ref 1 [ 938.530025] nvidia-fs:Pinning shadow buffer 7f5b596bd000 length = 1048576 [ 938.530037] nvidia-fs:Pinned Addr: 7f5b596bd000 256 pages for process id 8834 [ 938.530039] nvidia-fs:Page: ff919d80850f9540 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8000 page->flags: 57ffffc0000002 [ 938.530040] nvidia-fs:Page: ff919d80850f95c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8001 page->flags: 57ffffc0000002 [ 938.530041] nvidia-fs:Page: ff919d80850f9600 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8002 page->flags: 57ffffc0000002 [ 938.530043] nvidia-fs:Page: ff919d80850f9640 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8003 page->flags: 57ffffc0000002 [ 938.530044] nvidia-fs:Page: ff919d80850f9680 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8004 page->flags: 57ffffc0000002 [ 938.530045] nvidia-fs:Page: ff919d80850f96c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8005 page->flags: 57ffffc0000002 [ 938.530046] nvidia-fs:Page: ff919d80850f9700 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8006 page->flags: 57ffffc0000002 [ 938.530047] nvidia-fs:Page: ff919d80850f9740 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8007 page->flags: 57ffffc0000002 [ 938.530048] nvidia-fs:Page: ff919d80850f9780 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8008 page->flags: 57ffffc0000002 [ 938.530049] nvidia-fs:Page: ff919d80850f97c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8009 page->flags: 57ffffc0000002 [ 938.530050] nvidia-fs:Page: ff919d80850f9800 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe800a page->flags: 57ffffc0000002 [ 938.530050] nvidia-fs:Page: ff919d80850f9840 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe800b page->flags: 57ffffc0000002 [ 938.530051] nvidia-fs:Page: ff919d80850f9880 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe800c page->flags: 57ffffc0000002 [ 938.530052] nvidia-fs:Page: ff919d80850f98c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe800d page->flags: 57ffffc0000002 [ 938.530052] nvidia-fs:Page: ff919d80850f9900 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe800e page->flags: 57ffffc0000002 [ 938.530053] nvidia-fs:Page: ff919d80850f9940 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe800f page->flags: 57ffffc0000002 [ 938.530054] nvidia-fs:Page: ff919d80850f9980 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8010 page->flags: 57ffffc0000002 [ 938.530055] nvidia-fs:Page: ff919d80850f99c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8011 page->flags: 57ffffc0000002 [ 938.530056] nvidia-fs:Page: ff919d80850f9a00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8012 page->flags: 57ffffc0000002 [ 938.530056] nvidia-fs:Page: ff919d80850f9a40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8013 page->flags: 57ffffc0000002 [ 938.530057] nvidia-fs:Page: ff919d80850f9a80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8014 page->flags: 57ffffc0000002 [ 938.530058] nvidia-fs:Page: ff919d80850f9ac0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8015 page->flags: 57ffffc0000002 [ 938.530058] nvidia-fs:Page: ff919d80850f9b00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8016 page->flags: 57ffffc0000002 [ 938.530059] nvidia-fs:Page: ff919d80850f9b40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8017 page->flags: 57ffffc0000002 [ 938.530060] nvidia-fs:Page: ff919d80850f9b80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8018 page->flags: 57ffffc0000002 [ 938.530060] nvidia-fs:Page: ff919d80850f9bc0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8019 page->flags: 57ffffc0000002 [ 938.530061] nvidia-fs:Page: ff919d80850f9c00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe801a page->flags: 57ffffc0000002 [ 938.530062] nvidia-fs:Page: ff919d80850f9c40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe801b page->flags: 57ffffc0000002 [ 938.530063] nvidia-fs:Page: ff919d80850f9c80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe801c page->flags: 57ffffc0000002 [ 938.530064] nvidia-fs:Page: ff919d80850f9cc0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe801d page->flags: 57ffffc0000002 [ 938.530065] nvidia-fs:Page: ff919d80850f9d00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe801e page->flags: 57ffffc0000002 [ 938.530066] nvidia-fs:Page: ff919d80850f9d40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe801f page->flags: 57ffffc0000002 [ 938.530067] nvidia-fs:Page: ff919d80850f9d80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8020 page->flags: 57ffffc0000002 [ 938.530068] nvidia-fs:Page: ff919d80850f9dc0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8021 page->flags: 57ffffc0000002 [ 938.530069] nvidia-fs:Page: ff919d80850f9e00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8022 page->flags: 57ffffc0000002 [ 938.530071] nvidia-fs:Page: ff919d80850f9e40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8023 page->flags: 57ffffc0000002 [ 938.530072] nvidia-fs:Page: ff919d80850f9e80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8024 page->flags: 57ffffc0000002 [ 938.530073] nvidia-fs:Page: ff919d80850f9ec0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8025 page->flags: 57ffffc0000002 [ 938.530074] nvidia-fs:Page: ff919d80850fbc00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8026 page->flags: 57ffffc0000002 [ 938.530074] nvidia-fs:Page: ff919d80850fbc40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8027 page->flags: 57ffffc0000002 [ 938.530075] nvidia-fs:Page: ff919d80850fbc80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8028 page->flags: 57ffffc0000002 [ 938.530075] nvidia-fs:Page: ff919d80850fbcc0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8029 page->flags: 57ffffc0000002 [ 938.530076] nvidia-fs:Page: ff919d80850fbd00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe802a page->flags: 57ffffc0000002 [ 938.530077] nvidia-fs:Page: ff919d80850fbd40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe802b page->flags: 57ffffc0000002 [ 938.530077] nvidia-fs:Page: ff919d80850fbd80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe802c page->flags: 57ffffc0000002 [ 938.530078] nvidia-fs:Page: ff919d80850fbdc0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe802d page->flags: 57ffffc0000002 [ 938.530078] nvidia-fs:Page: ff919d80850fbe00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe802e page->flags: 57ffffc0000002 [ 938.530079] nvidia-fs:Page: ff919d80850fbe40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe802f page->flags: 57ffffc0000002 [ 938.530081] nvidia-fs:Page: ff919d80850fbe80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8030 page->flags: 57ffffc0000002 [ 938.530081] nvidia-fs:Page: ff919d80850fbec0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8031 page->flags: 57ffffc0000002 [ 938.530082] nvidia-fs:Page: ff919d80850fbf00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8032 page->flags: 57ffffc0000002 [ 938.530082] nvidia-fs:Page: ff919d80850fbf40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8033 page->flags: 57ffffc0000002 [ 938.530083] nvidia-fs:Page: ff919d80850fbf80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8034 page->flags: 57ffffc0000002 [ 938.530084] nvidia-fs:Page: ff919d80850fbfc0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8035 page->flags: 57ffffc0000002 [ 938.530084] nvidia-fs:Page: ff919d80850fc000 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8036 page->flags: 57ffffc0000002 [ 938.530085] nvidia-fs:Page: ff919d80850fc040 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8037 page->flags: 57ffffc0000002 [ 938.530085] nvidia-fs:Page: ff919d80850fc080 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8038 page->flags: 57ffffc0000002 [ 938.530086] nvidia-fs:Page: ff919d80850fc0c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8039 page->flags: 57ffffc0000002 [ 938.530087] nvidia-fs:Page: ff919d80850fc100 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe803a page->flags: 57ffffc0000002 [ 938.530088] nvidia-fs:Page: ff919d80850fc140 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe803b page->flags: 57ffffc0000002 [ 938.530090] nvidia-fs:Page: ff919d80850fc180 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe803c page->flags: 57ffffc0000002 [ 938.530091] nvidia-fs:Page: ff919d80850fc1c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe803d page->flags: 57ffffc0000002 [ 938.530092] nvidia-fs:Page: ff919d80850fc200 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe803e page->flags: 57ffffc0000002 [ 938.530092] nvidia-fs:Page: ff919d80850fc240 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe803f page->flags: 57ffffc0000002 [ 938.530093] nvidia-fs:Page: ff919d80850fc280 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8040 page->flags: 57ffffc0000002 [ 938.530094] nvidia-fs:Page: ff919d80850fc2c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8041 page->flags: 57ffffc0000002 [ 938.530094] nvidia-fs:Page: ff919d80850fc300 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8042 page->flags: 57ffffc0000002 [ 938.530095] nvidia-fs:Page: ff919d80850fc340 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8043 page->flags: 57ffffc0000002 [ 938.530095] nvidia-fs:Page: ff919d80850fc380 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8044 page->flags: 57ffffc0000002 [ 938.530096] nvidia-fs:Page: ff919d80850fc3c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8045 page->flags: 57ffffc0000002 [ 938.530098] nvidia-fs:Page: ff919d80850fc400 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8046 page->flags: 57ffffc0000002 [ 938.530099] nvidia-fs:Page: ff919d80850fc440 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8047 page->flags: 57ffffc0000002 [ 938.530099] nvidia-fs:Page: ff919d80850fc480 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8048 page->flags: 57ffffc0000002 [ 938.530100] nvidia-fs:Page: ff919d80850fc4c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8049 page->flags: 57ffffc0000002 [ 938.530101] nvidia-fs:Page: ff919d80850fc500 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe804a page->flags: 57ffffc0000002 [ 938.530101] nvidia-fs:Page: ff919d80850fc540 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe804b page->flags: 57ffffc0000002 [ 938.530102] nvidia-fs:Page: ff919d80850fc580 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe804c page->flags: 57ffffc0000002 [ 938.530103] nvidia-fs:Page: ff919d80850fc5c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe804d page->flags: 57ffffc0000002 [ 938.530103] nvidia-fs:Page: ff919d80850fc600 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe804e page->flags: 57ffffc0000002 [ 938.530104] nvidia-fs:Page: ff919d80850fc640 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe804f page->flags: 57ffffc0000002 [ 938.530104] nvidia-fs:Page: ff919d80850fc680 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8050 page->flags: 57ffffc0000002 [ 938.530106] nvidia-fs:Page: ff919d80850fc6c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8051 page->flags: 57ffffc0000002 [ 938.530107] nvidia-fs:Page: ff919d80850fc700 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8052 page->flags: 57ffffc0000002 [ 938.530108] nvidia-fs:Page: ff919d80850fc740 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8053 page->flags: 57ffffc0000002 [ 938.530108] nvidia-fs:Page: ff919d80850fc780 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8054 page->flags: 57ffffc0000002 [ 938.530109] nvidia-fs:Page: ff919d80850fc7c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8055 page->flags: 57ffffc0000002 [ 938.530110] nvidia-fs:Page: ff919d80850fc800 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8056 page->flags: 57ffffc0000002 [ 938.530111] nvidia-fs:Page: ff919d80850fc840 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8057 page->flags: 57ffffc0000002 [ 938.530112] nvidia-fs:Page: ff919d80850fc880 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8058 page->flags: 57ffffc0000002 [ 938.530113] nvidia-fs:Page: ff919d80850fc8c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8059 page->flags: 57ffffc0000002 [ 938.530114] nvidia-fs:Page: ff919d80850fc900 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe805a page->flags: 57ffffc0000002 [ 938.530115] nvidia-fs:Page: ff919d80850fc940 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe805b page->flags: 57ffffc0000002 [ 938.530116] nvidia-fs:Page: ff919d80850fc980 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe805c page->flags: 57ffffc0000002 [ 938.530117] nvidia-fs:Page: ff919d80850fc9c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe805d page->flags: 57ffffc0000002 [ 938.530117] nvidia-fs:Page: ff919d80850fca00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe805e page->flags: 57ffffc0000002 [ 938.530118] nvidia-fs:Page: ff919d80850fca40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe805f page->flags: 57ffffc0000002 [ 938.530118] nvidia-fs:Page: ff919d80850fca80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8060 page->flags: 57ffffc0000002 [ 938.530119] nvidia-fs:Page: ff919d80850fcac0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8061 page->flags: 57ffffc0000002 [ 938.530120] nvidia-fs:Page: ff919d80850fcb00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8062 page->flags: 57ffffc0000002 [ 938.530120] nvidia-fs:Page: ff919d80850fcb40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8063 page->flags: 57ffffc0000002 [ 938.530121] nvidia-fs:Page: ff919d80850fcb80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8064 page->flags: 57ffffc0000002 [ 938.530122] nvidia-fs:Page: ff919d80850fcbc0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8065 page->flags: 57ffffc0000002 [ 938.530122] nvidia-fs:Page: ff919d80850fcc00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8066 page->flags: 57ffffc0000002 [ 938.530123] nvidia-fs:Page: ff919d80850fcc40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8067 page->flags: 57ffffc0000002 [ 938.530123] nvidia-fs:Page: ff919d80850fcc80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8068 page->flags: 57ffffc0000002 [ 938.530125] nvidia-fs:Page: ff919d80850fccc0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8069 page->flags: 57ffffc0000002 [ 938.530126] nvidia-fs:Page: ff919d80850fcd00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe806a page->flags: 57ffffc0000002 [ 938.530126] nvidia-fs:Page: ff919d80850fcd40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe806b page->flags: 57ffffc0000002 [ 938.530127] nvidia-fs:Page: ff919d80850fcd80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe806c page->flags: 57ffffc0000002 [ 938.530128] nvidia-fs:Page: ff919d80850fcdc0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe806d page->flags: 57ffffc0000002 [ 938.530128] nvidia-fs:Page: ff919d80850fce00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe806e page->flags: 57ffffc0000002 [ 938.530129] nvidia-fs:Page: ff919d80850fce40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe806f page->flags: 57ffffc0000002 [ 938.530129] nvidia-fs:Page: ff919d80850fce80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8070 page->flags: 57ffffc0000002 [ 938.530131] nvidia-fs:Page: ff919d80850fcec0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8071 page->flags: 57ffffc0000002 [ 938.530132] nvidia-fs:Page: ff919d80850fcf00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8072 page->flags: 57ffffc0000002 [ 938.530134] nvidia-fs:Page: ff919d80850fcf40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8073 page->flags: 57ffffc0000002 [ 938.530135] nvidia-fs:Page: ff919d80850fcf80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8074 page->flags: 57ffffc0000002 [ 938.530136] nvidia-fs:Page: ff919d80850fcfc0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8075 page->flags: 57ffffc0000002 [ 938.530137] nvidia-fs:Page: ff919d80850fd000 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8076 page->flags: 57ffffc0000002 [ 938.530137] nvidia-fs:Page: ff919d80850fd040 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8077 page->flags: 57ffffc0000002 [ 938.530138] nvidia-fs:Page: ff919d80850fd080 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8078 page->flags: 57ffffc0000002 [ 938.530138] nvidia-fs:Page: ff919d80850fd0c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8079 page->flags: 57ffffc0000002 [ 938.530139] nvidia-fs:Page: ff919d80850fd100 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe807a page->flags: 57ffffc0000002 [ 938.530140] nvidia-fs:Page: ff919d80850fd140 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe807b page->flags: 57ffffc0000002 [ 938.530140] nvidia-fs:Page: ff919d80850fd180 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe807c page->flags: 57ffffc0000002 [ 938.530141] nvidia-fs:Page: ff919d80850fd1c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe807d page->flags: 57ffffc0000002 [ 938.530142] nvidia-fs:Page: ff919d80850fd200 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe807e page->flags: 57ffffc0000002 [ 938.530143] nvidia-fs:Page: ff919d80850fd240 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe807f page->flags: 57ffffc0000002 [ 938.530144] nvidia-fs:Page: ff919d80850fd280 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8080 page->flags: 57ffffc0000002 [ 938.530144] nvidia-fs:Page: ff919d80850fd2c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8081 page->flags: 57ffffc0000002 [ 938.530145] nvidia-fs:Page: ff919d80850fd300 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8082 page->flags: 57ffffc0000002 [ 938.530145] nvidia-fs:Page: ff919d80850fd340 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8083 page->flags: 57ffffc0000002 [ 938.530146] nvidia-fs:Page: ff919d80850fd380 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8084 page->flags: 57ffffc0000002 [ 938.530147] nvidia-fs:Page: ff919d80850fd3c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8085 page->flags: 57ffffc0000002 [ 938.530147] nvidia-fs:Page: ff919d80850fd400 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8086 page->flags: 57ffffc0000002 [ 938.530148] nvidia-fs:Page: ff919d80850fd440 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8087 page->flags: 57ffffc0000002 [ 938.530149] nvidia-fs:Page: ff919d80850fd480 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8088 page->flags: 57ffffc0000002 [ 938.530149] nvidia-fs:Page: ff919d80850fd4c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8089 page->flags: 57ffffc0000002 [ 938.530150] nvidia-fs:Page: ff919d80850fd500 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe808a page->flags: 57ffffc0000002 [ 938.530151] nvidia-fs:Page: ff919d80850fd540 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe808b page->flags: 57ffffc0000002 [ 938.530152] nvidia-fs:Page: ff919d80850fd580 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe808c page->flags: 57ffffc0000002 [ 938.530154] nvidia-fs:Page: ff919d80850fd5c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe808d page->flags: 57ffffc0000002 [ 938.530155] nvidia-fs:Page: ff919d80850fd600 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe808e page->flags: 57ffffc0000002 [ 938.530156] nvidia-fs:Page: ff919d80850fd640 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe808f page->flags: 57ffffc0000002 [ 938.530156] nvidia-fs:Page: ff919d80850fd680 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8090 page->flags: 57ffffc0000002 [ 938.530157] nvidia-fs:Page: ff919d80850fd6c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8091 page->flags: 57ffffc0000002 [ 938.530158] nvidia-fs:Page: ff919d80850fd700 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8092 page->flags: 57ffffc0000002 [ 938.530158] nvidia-fs:Page: ff919d80850fd740 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8093 page->flags: 57ffffc0000002 [ 938.530159] nvidia-fs:Page: ff919d80850fd780 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8094 page->flags: 57ffffc0000002 [ 938.530160] nvidia-fs:Page: ff919d80850fd7c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8095 page->flags: 57ffffc0000002 [ 938.530161] nvidia-fs:Page: ff919d80850fd800 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8096 page->flags: 57ffffc0000002 [ 938.530161] nvidia-fs:Page: ff919d80850fd840 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8097 page->flags: 57ffffc0000002 [ 938.530162] nvidia-fs:Page: ff919d80850fd880 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8098 page->flags: 57ffffc0000002 [ 938.530163] nvidia-fs:Page: ff919d80850fd8c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe8099 page->flags: 57ffffc0000002 [ 938.530163] nvidia-fs:Page: ff919d80850fd900 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe809a page->flags: 57ffffc0000002 [ 938.530164] nvidia-fs:Page: ff919d80850fd940 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe809b page->flags: 57ffffc0000002 [ 938.530164] nvidia-fs:Page: ff919d80850fd980 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe809c page->flags: 57ffffc0000002 [ 938.530165] nvidia-fs:Page: ff919d80850fd9c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe809d page->flags: 57ffffc0000002 [ 938.530166] nvidia-fs:Page: ff919d80850fda00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe809e page->flags: 57ffffc0000002 [ 938.530166] nvidia-fs:Page: ff919d80850fda40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe809f page->flags: 57ffffc0000002 [ 938.530167] nvidia-fs:Page: ff919d80850fda80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80a0 page->flags: 57ffffc0000002 [ 938.530168] nvidia-fs:Page: ff919d80850fdac0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80a1 page->flags: 57ffffc0000002 [ 938.530169] nvidia-fs:Page: ff919d80850fdb00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80a2 page->flags: 57ffffc0000002 [ 938.530169] nvidia-fs:Page: ff919d80850fdb40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80a3 page->flags: 57ffffc0000002 [ 938.530170] nvidia-fs:Page: ff919d80850fdb80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80a4 page->flags: 57ffffc0000002 [ 938.530171] nvidia-fs:Page: ff919d80850fdbc0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80a5 page->flags: 57ffffc0000002 [ 938.530171] nvidia-fs:Page: ff919d80850fdc00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80a6 page->flags: 57ffffc0000002 [ 938.530172] nvidia-fs:Page: ff919d80850fdc40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80a7 page->flags: 57ffffc0000002 [ 938.530173] nvidia-fs:Page: ff919d80850fdc80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80a8 page->flags: 57ffffc0000002 [ 938.530174] nvidia-fs:Page: ff919d80850fdcc0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80a9 page->flags: 57ffffc0000002 [ 938.530175] nvidia-fs:Page: ff919d80850fdd00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80aa page->flags: 57ffffc0000002 [ 938.530177] nvidia-fs:Page: ff919d80850fdd40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80ab page->flags: 57ffffc0000002 [ 938.530178] nvidia-fs:Page: ff919d80850fdd80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80ac page->flags: 57ffffc0000002 [ 938.530178] nvidia-fs:Page: ff919d80850fddc0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80ad page->flags: 57ffffc0000002 [ 938.530179] nvidia-fs:Page: ff919d80850fde00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80ae page->flags: 57ffffc0000002 [ 938.530180] nvidia-fs:Page: ff919d80850fde40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80af page->flags: 57ffffc0000002 [ 938.530180] nvidia-fs:Page: ff919d80850fde80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80b0 page->flags: 57ffffc0000002 [ 938.530181] nvidia-fs:Page: ff919d80850fdec0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80b1 page->flags: 57ffffc0000002 [ 938.530181] nvidia-fs:Page: ff919d80850fdf00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80b2 page->flags: 57ffffc0000002 [ 938.530182] nvidia-fs:Page: ff919d80850fdf40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80b3 page->flags: 57ffffc0000002 [ 938.530183] nvidia-fs:Page: ff919d80850fdf80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80b4 page->flags: 57ffffc0000002 [ 938.530183] nvidia-fs:Page: ff919d80850fdfc0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80b5 page->flags: 57ffffc0000002 [ 938.530184] nvidia-fs:Page: ff919d80850fe000 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80b6 page->flags: 57ffffc0000002 [ 938.530186] nvidia-fs:Page: ff919d80850fe040 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80b7 page->flags: 57ffffc0000002 [ 938.530187] nvidia-fs:Page: ff919d80850fe080 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80b8 page->flags: 57ffffc0000002 [ 938.530187] nvidia-fs:Page: ff919d80850fe0c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80b9 page->flags: 57ffffc0000002 [ 938.530188] nvidia-fs:Page: ff919d80850fe100 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80ba page->flags: 57ffffc0000002 [ 938.530189] nvidia-fs:Page: ff919d80850fe140 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80bb page->flags: 57ffffc0000002 [ 938.530189] nvidia-fs:Page: ff919d80850fe180 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80bc page->flags: 57ffffc0000002 [ 938.530190] nvidia-fs:Page: ff919d80850fe1c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80bd page->flags: 57ffffc0000002 [ 938.530190] nvidia-fs:Page: ff919d80850fe200 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80be page->flags: 57ffffc0000002 [ 938.530191] nvidia-fs:Page: ff919d80850fe240 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80bf page->flags: 57ffffc0000002 [ 938.530192] nvidia-fs:Page: ff919d80850fe280 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80c0 page->flags: 57ffffc0000002 [ 938.530192] nvidia-fs:Page: ff919d80850fe2c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80c1 page->flags: 57ffffc0000002 [ 938.530194] nvidia-fs:Page: ff919d80850fe300 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80c2 page->flags: 57ffffc0000002 [ 938.530195] nvidia-fs:Page: ff919d80850fe340 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80c3 page->flags: 57ffffc0000002 [ 938.530197] nvidia-fs:Page: ff919d80850fe380 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80c4 page->flags: 57ffffc0000002 [ 938.530198] nvidia-fs:Page: ff919d80850fe3c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80c5 page->flags: 57ffffc0000002 [ 938.530198] nvidia-fs:Page: ff919d80850fe400 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80c6 page->flags: 57ffffc0000002 [ 938.530199] nvidia-fs:Page: ff919d80850fe440 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80c7 page->flags: 57ffffc0000002 [ 938.530200] nvidia-fs:Page: ff919d80850fe480 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80c8 page->flags: 57ffffc0000002 [ 938.530200] nvidia-fs:Page: ff919d80850fe4c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80c9 page->flags: 57ffffc0000002 [ 938.530201] nvidia-fs:Page: ff919d80850fe500 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80ca page->flags: 57ffffc0000002 [ 938.530201] nvidia-fs:Page: ff919d80850fe540 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80cb page->flags: 57ffffc0000002 [ 938.530202] nvidia-fs:Page: ff919d80850fe580 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80cc page->flags: 57ffffc0000002 [ 938.530203] nvidia-fs:Page: ff919d80850fe5c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80cd page->flags: 57ffffc0000002 [ 938.530203] nvidia-fs:Page: ff919d80850fe600 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80ce page->flags: 57ffffc0000002 [ 938.530204] nvidia-fs:Page: ff919d80850fe640 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80cf page->flags: 57ffffc0000002 [ 938.530205] nvidia-fs:Page: ff919d80850fe680 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80d0 page->flags: 57ffffc0000002 [ 938.530206] nvidia-fs:Page: ff919d80850fe6c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80d1 page->flags: 57ffffc0000002 [ 938.530206] nvidia-fs:Page: ff919d80850fe700 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80d2 page->flags: 57ffffc0000002 [ 938.530207] nvidia-fs:Page: ff919d80850fe740 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80d3 page->flags: 57ffffc0000002 [ 938.530208] nvidia-fs:Page: ff919d80850fe780 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80d4 page->flags: 57ffffc0000002 [ 938.530208] nvidia-fs:Page: ff919d80850fe7c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80d5 page->flags: 57ffffc0000002 [ 938.530209] nvidia-fs:Page: ff919d80850fe800 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80d6 page->flags: 57ffffc0000002 [ 938.530209] nvidia-fs:Page: ff919d80850fe840 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80d7 page->flags: 57ffffc0000002 [ 938.530210] nvidia-fs:Page: ff919d80850fe880 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80d8 page->flags: 57ffffc0000002 [ 938.530211] nvidia-fs:Page: ff919d80850fe8c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80d9 page->flags: 57ffffc0000002 [ 938.530212] nvidia-fs:Page: ff919d80850fe900 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80da page->flags: 57ffffc0000002 [ 938.530213] nvidia-fs:Page: ff919d80850fe940 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80db page->flags: 57ffffc0000002 [ 938.530213] nvidia-fs:Page: ff919d80850fe980 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80dc page->flags: 57ffffc0000002 [ 938.530215] nvidia-fs:Page: ff919d80850fe9c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80dd page->flags: 57ffffc0000002 [ 938.530216] nvidia-fs:Page: ff919d80850fea00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80de page->flags: 57ffffc0000002 [ 938.530217] nvidia-fs:Page: ff919d80850fea40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80df page->flags: 57ffffc0000002 [ 938.530218] nvidia-fs:Page: ff919d80850fea80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80e0 page->flags: 57ffffc0000002 [ 938.530218] nvidia-fs:Page: ff919d80850feac0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80e1 page->flags: 57ffffc0000002 [ 938.530219] nvidia-fs:Page: ff919d80850feb00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80e2 page->flags: 57ffffc0000002 [ 938.530220] nvidia-fs:Page: ff919d80850feb40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80e3 page->flags: 57ffffc0000002 [ 938.530221] nvidia-fs:Page: ff919d80850feb80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80e4 page->flags: 57ffffc0000002 [ 938.530222] nvidia-fs:Page: ff919d80850febc0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80e5 page->flags: 57ffffc0000002 [ 938.530222] nvidia-fs:Page: ff919d80850fec00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80e6 page->flags: 57ffffc0000002 [ 938.530223] nvidia-fs:Page: ff919d80850fec40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80e7 page->flags: 57ffffc0000002 [ 938.530224] nvidia-fs:Page: ff919d80850fec80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80e8 page->flags: 57ffffc0000002 [ 938.530224] nvidia-fs:Page: ff919d80850fecc0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80e9 page->flags: 57ffffc0000002 [ 938.530225] nvidia-fs:Page: ff919d80850fed00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80ea page->flags: 57ffffc0000002 [ 938.530226] nvidia-fs:Page: ff919d80850fed40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80eb page->flags: 57ffffc0000002 [ 938.530226] nvidia-fs:Page: ff919d80850fed80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80ec page->flags: 57ffffc0000002 [ 938.530227] nvidia-fs:Page: ff919d80850fedc0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80ed page->flags: 57ffffc0000002 [ 938.530227] nvidia-fs:Page: ff919d80850fee00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80ee page->flags: 57ffffc0000002 [ 938.530228] nvidia-fs:Page: ff919d80850fee40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80ef page->flags: 57ffffc0000002 [ 938.530229] nvidia-fs:Page: ff919d80850fee80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80f0 page->flags: 57ffffc0000002 [ 938.530230] nvidia-fs:Page: ff919d80850feec0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80f1 page->flags: 57ffffc0000002 [ 938.530231] nvidia-fs:Page: ff919d80850fef00 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80f2 page->flags: 57ffffc0000002 [ 938.530232] nvidia-fs:Page: ff919d80850fef40 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80f3 page->flags: 57ffffc0000002 [ 938.530233] nvidia-fs:Page: ff919d80850fef80 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80f4 page->flags: 57ffffc0000002 [ 938.530234] nvidia-fs:Page: ff919d80850fefc0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80f5 page->flags: 57ffffc0000002 [ 938.530234] nvidia-fs:Page: ff919d80850ff000 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80f6 page->flags: 57ffffc0000002 [ 938.530235] nvidia-fs:Page: ff919d80850ff040 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80f7 page->flags: 57ffffc0000002 [ 938.530236] nvidia-fs:Page: ff919d80850ff080 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80f8 page->flags: 57ffffc0000002 [ 938.530237] nvidia-fs:Page: ff919d80850ff0c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80f9 page->flags: 57ffffc0000002 [ 938.530239] nvidia-fs:Page: ff919d80850ff100 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80fa page->flags: 57ffffc0000002 [ 938.530240] nvidia-fs:Page: ff919d80850ff140 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80fb page->flags: 57ffffc0000002 [ 938.530240] nvidia-fs:Page: ff919d80850ff180 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80fc page->flags: 57ffffc0000002 [ 938.530241] nvidia-fs:Page: ff919d80850ff1c0 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80fd page->flags: 57ffffc0000002 [ 938.530241] nvidia-fs:Page: ff919d80850ff200 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80fe page->flags: 57ffffc0000002 [ 938.530242] nvidia-fs:Page: ff919d80850ff240 , nvfs_mgroup: 000000000b447889, base_index: 164a79fe8 page-index: 164a79fe80ff page->flags: 57ffffc0000002 [ 938.530245] nvidia-fs:successfully pinned end fence address : 1cbe000, end_fence_page : ff919d8083d19640 offset in page : 0x in kernel [ 938.530246] nvfs_pin_gpu_pages [ 938.530247] nvidia-fs:IO Transit requested from FREE->INIT nvfsio :00000000b243e4f7 [ 938.530248] nvidia-fs:IO Transit success FREE->INIT nvfsio :00000000b243e4f7 [ 938.530248] nvidia-fs:nvfs_pin_gpu_pages:1245 IO State moved from FREE to INIT [ 938.530249] nvidia-fs:gpu_addr 0x7f5b4c600000 cpu_addr 0x7f5b596bd000 gpu_buf_len 1048576 [ 938.530250] nvidia-fs:Invoking p2p_get_pages pages (0x7f5b4c600000 - 0x7f5b4c6fffff) rounded size 100000 [ 938.530330] nvidia-fs:GPU page table entries: 16 [ 938.530331] nvidia-fs:GPU Physical page[0]=0x00001e6000400000 [ 938.530331] nvidia-fs:GPU Physical page[1]=0x00001e6000410000 [ 938.530332] nvidia-fs:GPU Physical page[2]=0x00001e6000420000 [ 938.530332] nvidia-fs:GPU Physical page[3]=0x00001e6000430000 [ 938.530332] nvidia-fs:GPU Physical page[4]=0x00001e6000440000 [ 938.530333] nvidia-fs:GPU Physical page[5]=0x00001e6000450000 [ 938.530333] nvidia-fs:GPU Physical page[6]=0x00001e6000460000 [ 938.530333] nvidia-fs:GPU Physical page[7]=0x00001e6000470000 [ 938.530334] nvidia-fs:GPU Physical page[8]=0x00001e6000480000 [ 938.530334] nvidia-fs:GPU Physical page[9]=0x00001e6000490000 [ 938.530335] nvidia-fs:GPU Physical page[10]=0x00001e60004a0000 [ 938.530336] nvidia-fs:GPU Physical page[11]=0x00001e60004b0000 [ 938.530337] nvidia-fs:GPU Physical page[12]=0x00001e60004c0000 [ 938.530338] nvidia-fs:GPU Physical page[13]=0x00001e60004d0000 [ 938.530338] nvidia-fs:GPU Physical page[14]=0x00001e60004e0000 [ 938.530339] nvidia-fs:GPU pages pinned successfully gpu_info 00000000c310d357 [ 938.530339] nvidia-fs:IO Transit requested from INIT->READY nvfsio :00000000b243e4f7 [ 938.530340] nvidia-fs:IO Transit success INIT->READY nvfsio :00000000b243e4f7 [ 938.530341] nvidia-fs:mmap gpu_info=00000000c310d357 mgroup 000000000b447889 pdevinfo :0000:8b:00.0 ref 2 IO state READY [ 938.530342] nvidia-fs:nvfs ioctl map success [ 938.540761] nvidia-fs:NVFS VMA close vma:00000000dd88f1d2 nvfs_mgroup 000000000b447889 [ 938.540768] nvidia-fs:munmap invoked - IO state TERMINATED 5 5 [ 938.540769] nvidia-fs:*****munmap invoked - nvfs_mgroup ref 2 mgroup 000000000b447889 [ 938.540771] nvidia-fs:nvfs_mgroup_put called 2 [ 938.540771] nvidia-fs:nvfs_mgroup_put called 1 [ 938.540772] nvidia-fs:nvfs_free_gpu_info state = TERMINATED [ 938.540773] nvidia-fs:unpin device physical pages [ 938.540847] nvidia-fs:irq_disabled = 0, in intr = 0, in atomic = 0, in nmi = 0 current->mm = 0 [ 938.541068] nvidia-fs:freeing base_index 0(ref:0) found [ 938.541543] nvidia-fs:nvfs ioctl remove invoked [ 938.541544] nvidia-fs:nvfs_remove [ 938.541571] nvidia-fs:nvfs_close [ 938.541572] nvidia-fs:nvfs ioctl remove invoked [ 938.541573] nvidia-fs:nvfs_remove [ 938.541591] nvidia-fs:nvfs_close [ 938.541592] nvidia-fs:nvfs ioctl remove invoked [ 938.541592] nvidia-fs:nvfs_remove [ 938.541610] nvidia-fs:nvfs_close [ 938.541611] nvidia-fs:nvfs ioctl remove invoked [ 938.541611] nvidia-fs:nvfs_remove [ 938.541628] nvidia-fs:nvfs_close [ 938.541629] nvidia-fs:nvfs ioctl remove invoked [ 938.541629] nvidia-fs:nvfs_remove [ 938.541645] nvidia-fs:nvfs_close [ 938.541647] nvidia-fs:nvfs ioctl remove invoked [ 938.541647] nvidia-fs:nvfs_remove [ 938.541676] nvidia-fs:nvfs_close [ 938.541677] nvidia-fs:nvfs ioctl remove invoked [ 938.541679] nvidia-fs:nvfs_remove [ 938.541696] nvidia-fs:nvfs_close [ 938.541697] nvidia-fs:nvfs ioctl remove invoked [ 938.541698] nvidia-fs:nvfs_remove [ 938.541714] nvidia-fs:nvfs_close [ 938.541716] nvidia-fs:nvfs ioctl remove invoked [ 938.541716] nvidia-fs:nvfs_remove [ 938.541734] nvidia-fs:nvfs_close [ 938.541736] nvidia-fs:nvfs ioctl remove invoked [ 938.541737] nvidia-fs:nvfs_remove [ 938.541754] nvidia-fs:nvfs_close [ 938.541755] nvidia-fs:nvfs ioctl remove invoked [ 938.541756] nvidia-fs:nvfs_remove [ 938.541771] nvidia-fs:nvfs_close [ 938.541772] nvidia-fs:nvfs ioctl remove invoked [ 938.541773] nvidia-fs:nvfs_remove [ 938.541791] nvidia-fs:nvfs_close [ 938.541792] nvidia-fs:nvfs ioctl remove invoked [ 938.541793] nvidia-fs:nvfs_remove [ 938.541810] nvidia-fs:nvfs_close [ 938.541812] nvidia-fs:nvfs ioctl remove invoked [ 938.541813] nvidia-fs:nvfs_remove [ 938.541827] nvidia-fs:nvfs_close [ 938.541829] nvidia-fs:nvfs ioctl remove invoked [ 938.541829] nvidia-fs:nvfs_remove [ 938.541847] nvidia-fs:nvfs_close [ 938.541848] nvidia-fs:nvfs ioctl remove invoked [ 938.541850] nvidia-fs:nvfs_remove [ 938.541866] nvidia-fs:Unregistering dma ops and nvidia p2p ops [ 938.541867] nvidia-fs:nvfs_close

glingyan commented 3 months ago

root cause from cufile.log [root@localhost samples]# ./cufile_sample_031 t1 t2 0 file register error: GPUDirect Storage not supported on current file

glingyan commented 3 months ago

NVMe Driver not registered with nvidia-fs