ERROR: unexpected error - Unable to load libhdfs

[oendnsk675]

Bug Report

Issue name

DVC tool to push data to HDFS

Command

dvc push -v

Description

I have a problem in pushing data to HDFS, previously I installed HDFS with DVC on WSL Ubuntu, and I got an error like this:

2024-10-18 19:08:32,210 ERROR: unexpected error - Unable to load libhdfs dlopen(libhdfs.so) failed: libhdfs.so: cannot open shared object file: No such file or directory dlopen(./libhdfs.so) failed: ./libhdfs.so: cannot open shared object file: No such file or directory dlopen(/usr/local/hadoop/lib/native/libhdfs.so) failed: /usr/local/hadoop/lib/native/libhdfs.so: cannot open shared object file: No such file or directory dlopen(/usr/local/hadoop/lib/native/libhdfs.so) failed: /usr/local/hadoop/lib/native/libhdfs.so: cannot open shared object file: No such file or directory

Expected

Data should be pushed to HDFS.

Output of dvc doctor

$ dvc doctor

DVC version: 3.55.2 (pip)
-------------------------
Platform: Python 3.10.12 on Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
Subprojects:
        dvc_data = 3.16.6
        dvc_objects = 5.1.0
        dvc_render = 1.0.2
        dvc_task = 0.40.2
        scmrepo = 3.3.8
Supports:
        hdfs (fsspec = 2024.9.0, pyarrow = 17.0.0),
        http (aiohttp = 3.10.10, aiohttp-retry = 2.8.3),
        https (aiohttp = 3.10.10, aiohttp-retry = 2.8.3)
Config:
        Global: /home/oslab/.config/dvc
        System: /etc/xdg/dvc
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/sde
Caches: local
Remotes: hdfs
Workspace directory: ext4 on /dev/sde
Repo: dvc, git
Repo.site_cache_dir: /var/tmp/dvc/repo/19a167349a3a91d6f12dd887c9c8f42

[shcheklein]

Tbh, seem like some dependencies issue on the system. Could you read around on how to install libhdfs and make sure that it's being picked up please