search

OFS / opae-sdk

Open Programmable Acceleration Engine
https://ofs.github.io
BSD 3-Clause "New" or "Revised" License
251 stars 83 forks source link

ASE is not coming up with OFS. #2360

Closed saitej25 closed 9 months ago

saitej25 commented 2 years ago

After bringing up ASE simulation on ofs-afu/samples and running the host code is giving me these issues.

I'm using an Ubuntu machine with Questasim as simulator.

Error: init.c:205:opae_init() ERROR : WITH_ASE was set, but could not find opae_ase.cfg file AFC not found.

tswhison commented 2 years ago

Sorry that you're having trouble. We're not familiar with the path 'ofs-afu/samples'. Could you mention a little more about the packages you're attempting to use, eg what collaterals you downloaded and from where?

saitej25 commented 2 years ago

OFS is an early access project we received from Intel. It has open-source sources code for both BBS (Blue bitstream) and GBS (Green bitstream). Reference: https://www.intel.com/content/www/us/en/products/details/fpga/platforms/pac/open-fpga-stack.html Samples link: https://github.com/otcshare/ofs-afu/tree/master/samples

michael-adler commented 2 years ago

Was your OPAE SDK built with ASE support (-DOPAE_BUILD_SIM=1)? If not, that could be your problem.

If you did build with ASE support:

If you look in https://github.com/OPAE/opae-sdk/blob/master/opae-libs/libopae-c/init.c you will see that find_ase_cfg() tries many places to find opae_ase.cfg. The first place it looks is the configured installation path when the library was built. Did you move a compiled installation tree to a new location? It also tries the path to the source tree in case you built the library from source and are running out of the installation directory.

Failing that search, it looks in a few directories relative to your $HOME and also a few system locations. Those paths are all near the top of init.c.

There is a copy of opae_ase.cfg in https://github.com/OPAE/opae-sim/blob/master/ase/opae_ase.cfg.

saitej25 commented 2 years ago

Hello @michael-adler , Thank you for the response. I'm building my OPAE with "-DOPAE_BUILD_SIM=1" . This is how I'm able to bring up the ASE Client side working. I can also locate the opae_ase.cfg files in the simulation work directory. This issue comes when I run the host code (Server code).

PS: I'm using https://github.com/otcshare/ofs-fim/blob/master/ci/build_opae.sh script to build my OPAE. I suppose this satisfies all flags to build the OPAE for ASE too.

michael-adler commented 2 years ago

Only the host code follows this path, so the error can only be there.

Is it possible that you have two copies of OPAE compiled on the system and that the host code side of your simulation is using libraries compiled without ASE support?

Failing that, if you step through the host startup in a debugger you should be able to see the paths tested in init.c, which should make it clear where it is looking in your environment and why the file isn't found.

saitej25 commented 2 years ago

I do have two OPAE compiled on system and yes one is without ASE. The one without ASE is on /opt and the one with ASE is on my user space /home/user/ (Local install). However I don't export OPAE in /opt paths. Is there a better way to do this?

michael-adler commented 2 years ago

If you run:

ldd <path to your binary>

you will probably find that the resolved version of libopae-c.so is in /opt. Set up your paths so your build in /home is found first.

saitej25 commented 2 years ago

Here's what I see doing a ldd to the binary. Before this I also gave export LD_PRELOAD=/scratch/stbodank/IOFS_S10NX_porting/install-opae-sdk/lib/libopae-c-ase.so

Logs: stbodank@compute2:/scratch/stbodank/intel-ofs/ofs-afu-master/samples/hello_afu/sw$ ldd hello_afu init.c:205:opae_init() ERROR : WITH_ASE was set, but could not find opae_ase.cfg file linux-vdso.so.1 (0x00007ffd8c79d000) /scratch/stbodank/IOFS_S10NX_porting/install-opae-sdk/lib/libopae-c-ase.so (0x00007ffadf3b5000) libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007ffadf1ae000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ffadedbd000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ffadebb9000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ffade99a000) libjson-c.so.3 => /lib/x86_64-linux-gnu/libjson-c.so.3 (0x00007ffade78f000) /lib64/ld-linux-x86-64.so.2 (0x00007ffadf7d5000)

michael-adler commented 2 years ago

Use LD_LIBRARY_PATH

That's also the wrong .so, at least at the beginning of the run.

saitej25 commented 2 years ago

@michael-adler I've tried it.. Here are my observations.

stbodank@compute2: export LD_LIBRARY_PATH=/scratch/stbodank/IOFS_S10NX_porting/install-opae-sdk/lib/

stbodank@compute2:/scratch/stbodank/intel-ofs/ofs-afu-master/samples/hello_afu/sw$ ldd hello_afu
linux-vdso.so.1 (0x00007fffad865000) libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f88b8f2b000) libopae-c-ase.so.2 => /scratch/stbodank/IOFS_S10NX_porting/install-opae-sdk/lib/libopae-c-ase.so.2 (0x00007f88b8d0e000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f88b891d000) /lib64/ld-linux-x86-64.so.2 (0x00007f88b9335000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f88b8719000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f88b84fa000) libjson-c.so.3 => /lib/x86_64-linux-gnu/libjson-c.so.3 (0x00007f88b82ef000)

stbodank@compute2:/scratch/stbodank/intel-ofs/ofs-afu-master/samples/hello_afu/sw$ ./hello_afu init.c:205:opae_init() ERROR : WITH_ASE was set, but could not find opae_ase.cfg file AFC not found.

michael-adler commented 2 years ago

Use gdb, break in find_ase_cfg() and step a little bit. (You can see the source in the first message in this chain.) The first path it sets up is the installation tree. If you are running the library you think you are and it is installed properly, it should find:

/scratch/stbodank/IOFS_S10NX_porting/install-opae-sdk/share/opae/ase/opae_ase.cfg

It will probably be pointing somewhere else, or perhaps the cfg isn't there. Following the search in the code will likely get you running faster than my attempt at remote debugging.