Cannot run some examples

[alewarne]

While the mnist convolutional network demo runs for me I get an error on the others, for example the max pooling example. It seems that there is a file missing:

IOError: [Errno 2] No such file or directory: 'maxpoolMNIST_files/ir/ir-cluster_0_XlaCompiledKernel_true__XlaNumeConstantArgs_0_XlaNumResourceArgs_0_module-with-opt.ll'

[SumantSakhalkar]

I'm facing similar issues. Would you be able to tell me how you solved your issue?

[danielholanda]

Hi Sumant,

I'm not sure if this will solve your problem, but a good practice is to always make sure that you can run the example with python before trying to use LeFlow.

python name_of_example.py instead of ../../src/leflow name_of_example.py

If there is something wrong with the example, the error message from LeFlow might not be very useful. In the maxpool example, not having matplotlib installed, for example, might make LeFlow fail and not output a really useful error message.

[SumantSakhalkar]

Hi Daniel,

Thank you so much for your quick response. I did realize my mistake, I did not check the readme file in the examples directory. I used a clean Legup-VM and skipped the installation of the code dependencies like matplotlib. I have closed the issue.

Thank you so much for such an interesting project. I am working on my Masters Project and I wanted to implement Yolov3 or MobileNets on my Cyclone V DE0-CV board. Could you give me any advice? I did read about the issues where I cannot use large networks in my design as it is mapped to on-chip memories and you can run out of it easily. I also know for a fact that you can only input by creating .mif files or by editing the RTL but there is very little documentation on that and I am a little confused on how the code takes in this .mif file. Is there any publicly available documentation for the aforementioned issues? Also, would I need to add this board to the /boards directory or can I just create Verilog for Cyclone V DE1-Soc and implement it on the DE-0 board?

I apologize for so many questions and I really appreciate all your help. Thanks in advance.

Thanks and regards, Sumant Sakhalkar

---

From: Daniel Holanda Noronha notifications@github.com Sent: Friday, June 12, 2020 10:29 AM To: danielholanda/LeFlow LeFlow@noreply.github.com Cc: Sakhalkar,Sumant Sudhir Sumant.SAKHALKAR@colostate.edu; Comment comment@noreply.github.com Subject: Re: [danielholanda/LeFlow] Cannot run some examples (#25)

Hi Sumant,

I'm not sure if this will solve your problem, but a good practice is to always make sure that you can run the example with python before trying to use LeFlow.

python name_of_example.py instead of ../../src/leflow name_of_example.py

If there is something wrong with the example, the error message from LeFlow might not be very useful. In the maxpool example, not having matplotlib installed, for example, might make LeFlow fail and not output a really useful error message.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdanielholanda%2FLeFlow%2Fissues%2F25%23issuecomment-643370552&data=02%7C01%7CSumant.SAKHALKAR%40colostate.edu%7Ce1132d183ee64fa9ac4908d80eedba24%7Cafb58802ff7a4bb1ab21367ff2ecfc8b%7C0%7C0%7C637275761482851998&sdata=QmKiewi4aB%2FJq2e92M83Y3kshIoafYDf%2FygBVAmq7Ms%3D&reserved=0, or unsubscribehttps://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALHDBPFAZ7VG7FZUPNCRAVDRWJJVFANCNFSM4JBJD3BQ&data=02%7C01%7CSumant.SAKHALKAR%40colostate.edu%7Ce1132d183ee64fa9ac4908d80eedba24%7Cafb58802ff7a4bb1ab21367ff2ecfc8b%7C0%7C0%7C637275761482861992&sdata=24km0k79qZ8osmULO8htjtAPwYKE30BNmv8po4y5wdU%3D&reserved=0.

[danielholanda]

Hi Sumant,

1 - Unfortunately, LeFlow will not work well with Yolov3/MobileNets, as using on-chip memories for both activations and weights is not feasible for large networks.

2 - Some of the examples use the "mif.createMem()" function, which maps TensorFlow weights to mif files. Those files are then used to initialize on-chip memories during simulation/synthesis.

3 - If you want to use external signals as an input you can modify the generated Verilog to manually insert the values into the on-chip memories before running the network. This is a manual process and requires understanding the generated Verilog (which is not that easy). This is certainly doable, but I don't have any good examples to help with this.

3 - I'm not sure if LegUp supports the DE0. A trick that you can try is to generate the Verilog for a board that has no floating-point DSPs (assuming that the DE0 has no DSPs) and copy and paste the generated Verilog to a DE0 project.

Hope this helps

[SumantSakhalkar]

Hi Daniel,

This helps a lot. Thank you very much. I just wanted to ask are there any new CNN applications that I can take a look into that can be feasible with Leflow? I would be glad to be of help and would love to contribute to this project. I'm a novice at HLS tools though I have worked with a few of the commercial tools like Catapult and Vivado HLS as well as a few academic tools like Legup 4.0

Thanks and regards, Sumant Sakhalkar

---

From: Daniel Holanda Noronha notifications@github.com Sent: Friday, June 12, 2020 12:31 PM To: danielholanda/LeFlow LeFlow@noreply.github.com Cc: Sakhalkar,Sumant Sudhir Sumant.SAKHALKAR@colostate.edu; Comment comment@noreply.github.com Subject: Re: [danielholanda/LeFlow] Cannot run some examples (#25)

Hi Sumant,

1 - Unfortunately, LeFlow will not work well with Yolov3/MobileNets, as using on-chip memories for both activations and weights is not feasible for large networks.

2 - Some of the examples use the "mif.createMem()" function, which maps TensorFlow weights to mif files. Those files are then used to initialize on-chip memories during simulation/synthesis.

3 - If you want to use external signals as an input you can modify the generated Verilog to manually insert the values into the on-chip memories before running the network. This is a manual process and requires understanding the generated Verilog (which is not that easy). This is certainly doable, but I don't have any good examples to help with this.

3 - I'm not sure if LegUp supports the DE0. A trick that you can try is to generate the Verilog for a board that has no floating-point DSPs (assuming that the DE0 has no DSPs) and copy and paste the generated Verilog to a DE0 project.

Hope this helps

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdanielholanda%2FLeFlow%2Fissues%2F25%23issuecomment-643424462&data=02%7C01%7CSumant.SAKHALKAR%40colostate.edu%7C55a6dd57100642a560f108d80efec457%7Cafb58802ff7a4bb1ab21367ff2ecfc8b%7C0%7C1%7C637275834669416868&sdata=aHIy3HduuBAkaFnhY%2Fn9UjF1AgWGbyYgyB4hgxoddBs%3D&reserved=0, or unsubscribehttps://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALHDBPAHUM3JGHVJZ725CBDRWJX6RANCNFSM4JBJD3BQ&data=02%7C01%7CSumant.SAKHALKAR%40colostate.edu%7C55a6dd57100642a560f108d80efec457%7Cafb58802ff7a4bb1ab21367ff2ecfc8b%7C0%7C1%7C637275834669416868&sdata=NTReiPEK0Kx%2B%2FNs%2FVqgTLEZTuHBeKZa8Tlh8R8h%2FJ8g%3D&reserved=0.

[danielholanda]

Hi Sumant,

All public examples I have are already either in the examples folder or scattered through the different Github Issues. That being said, adding support to new models is not a priority now. If you want to contribute to the project there are quite a few things that you could do that are not related to supporting new models. Some of those include adding fixed-point support, changing XLA to dump LLVM IR that is a little bit more hardware friendly, and adding support for the use of external memories/interfaces.

[SumantSakhalkar]

Hi Daniel,

I deeply apologize for the late response and thank you for getting back to me.

That sounds really interesting. Adding support for the use of external memories is really interesting and would be something I can look into. I'm assuming I need to understand how the RTL is generated in a little more depth. You also mentioned that you want to make changes XLA architecture to dump LLVM IR that is a little bit more hardware friendly. Could you elaborate a little on that? I know that Leflow cannot generate the Verilog if there are vectorized instructions, is that what you are specifically into? Id love to have more insights from you.

I really appreciate this greatly.

Thanks and regards, Sumant Sakhalkar

---

From: Daniel Holanda Noronha notifications@github.com Sent: Monday, June 15, 2020 11:01 AM To: danielholanda/LeFlow LeFlow@noreply.github.com Cc: Sakhalkar,Sumant Sudhir Sumant.SAKHALKAR@colostate.edu; Comment comment@noreply.github.com Subject: Re: [danielholanda/LeFlow] Cannot run some examples (#25)

Hi Sumant,

All public examples I have are already either in the examples folder or scattered through the different Github Issues. That being said, adding support to new models is not a priority now. If you want to contribute to the project there are quite a few things that you could do that are not related to supporting new models. Some of those include adding fixed-point support, changing XLA to dump LLVM IR that is a little bit more hardware friendly, and adding support for the use of external memories/interfaces.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdanielholanda%2FLeFlow%2Fissues%2F25%23issuecomment-644254647&data=02%7C01%7CSumant.SAKHALKAR%40colostate.edu%7C193f849cef934b3c040d08d8114dbaf7%7Cafb58802ff7a4bb1ab21367ff2ecfc8b%7C0%7C0%7C637278372838675085&sdata=CVmtr%2Ffv0DyYyYnoanr91345kAdyxakMWD8ZvgAYD%2B0%3D&reserved=0, or unsubscribehttps://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALHDBPAAZVL22LZ4EYSNJZDRWZHWDANCNFSM4JBJD3BQ&data=02%7C01%7CSumant.SAKHALKAR%40colostate.edu%7C193f849cef934b3c040d08d8114dbaf7%7Cafb58802ff7a4bb1ab21367ff2ecfc8b%7C0%7C0%7C637278372838675085&sdata=G0KyhDooyKXpTrk17jgOdCgbrY4kI9qnb%2FTIwEqYENM%3D&reserved=0.

[danielholanda]

Hi Sumant,

Sorry for the delay.

"Making XLA dump LLVM IR that is a little bit more hardware friendly" means dumping LLVM IR that is in a better position to be used by Legup, resulting in a more efficient hardware. As an example, you could reorder the computations in a convolution to avoid reading the same value from dual-port rams multiple times. However, to make this really good, I would love to see someone being able to reuse the HW units. LeFlow currently generates a tiny circuit for each NN layer, which is not scalable. Modifying XLA to dump IR that would result in a Verilog that reuses HW units would be a lot of work, but would certainly make a big difference.