Installation makefile is not working for me

[kashperanto]

Hey, thanks again for putting this out there. Unfortunately I'm once again at a loss to determine why the makefile is not working for me.

I cloned master and attempted to run make, but it appears to do nothing. Do I need to set up some environment variables or something?

I'm running GNU Make 4.1

gcc version 7.4.0

Also, I'm running this on Ubuntu 18.04 on Windows Subsystem for Linux. Unless you're doing something crazy with assembly code I doubt the WSL part is important.

[aiobofh]

@kashperanto hi again.

Are you sure you donät get a build? - The make system is pretty quiet. It just speak out if something were to go wrong. :D

If you run "make check" it should build the tools and run the regression tests. (might require gcov)

... Or if you go to the src folder and make the tools, then go to the test folder and run "make" there, you should get a bunch of dotts (one for each check).

[aiobofh]

... there is also an example folder that should build and run checks.

[aiobofh]

@kashperanto if you want to see what the Makefile is doing just add Q= after the make-command. Like so:

$ make Q=

Then you will see the commands executed, if you were missing them :D

The install target makes an installation in /usr/local/bin here the tools will be installed. They're all 3-letter long starting with "t" :D

[kashperanto]

Ah, you are correct. I noticed the Q variable after my first attempt, and when it is already built it outputs nothing even with Q=

I did a quick make clean and then I saw good output, but I think something might be wrong with the PREFIX option:

$ sudo make install
mkdir -p REFIX/bin && \
install -s src/tcg REFIX/bin/tcg && \
install -s src/tam REFIX/bin/tam && \
install -s src/tmg REFIX/bin/tmg && \
install -s src/tsg REFIX/bin/tsg && \
install -s src/ttg REFIX/bin/ttg && \
mkdir -p REFIX/include/tarsio && \
install include/tarsio.mk REFIX/include/tarsio/tarsio.mk && \
install include/coverage.mk REFIX/include/tarsio/coverage.mk && \
install include/tarsio.h REFIX/include/tarsio/tarsio.h

I end up with a new folder in the tarsio/ directory called REFIX, which has bin/ and include/ folders with the expected files.

I think this is a variable name expansion issue. I changed every instance of $PREFIX to ${PREFIX} and it installed the files to my /bin directory. Looking at the README I think you intend the default behavior to install to /usr/, so I also added "export PREFIX:=/usr" and everything seemed to be fine.

I can put together a pull request, or you could make these changes how you would like. I have never done a pull request on github before, so that will take a few minutes to make sure I am doing everything right.

[aiobofh]

haha :) right. I did push some stuff an hour ago, that fix that problem.

[aiobofh]

If you start out making something based on the example-folder and fiddle around with that Makefile I think you might get going.

I'm currently working on more flexibility for e.g. out-of-source builds and such with a better .mk-file that one could include in ones projects Makefile. But I think everything you need to know is in the example/Makefile. Having a .mk-file for inclusion is just sugar and nice-to-have.

I'm just done porting some cutest-tests into Tarsio-checks instead in a small project i have here on Github as well. The bltools. Have not pushed yet tho. But that could be a nice example too once it's up.

[aiobofh]

@kashperanto let me know how things are going! :) also I pushed the refactored tests for bltools, you can use this as an example too. It has separate src and test folders and make use of an installed tarsio-tool-chain. It's here https://github.com/aiobofh/bltools

Especially look in the test/Makefile for tips and tricks...

B.t.w. I pushed som other changes to Tarsio where I moved the old tarsio.mk and coverage.mk, and made a new tarsio.mk that is a bit cleaner. That's the one used in bltools.

[aiobofh]

@kashperanto ... also. If you feel like discussing things in a faster way, head over to IRC (ircs://irc.aio.nu:6666/tarsio) which is a bit neater than GitHub issues :)

[kashperanto]

Hey, everything is working so far, I just need to take some time to get up to speed on the makefiles. Seeing that full project should be a help; I might get time this weekend to look at it in more detail.

I did get an android irc client recently, but ran into some issues trying to get myself set up (I need to get familiar with it again). Unfortunately everywhere I have ever worked locks the network down to basically only https unless you have a good (business) reason to go messing around with the network.

[aiobofh]

Awesome news! Enjoy! Don't hasitate to ping me if you get stuck.

Right... There are web-based clients. Like this one https://kiwiirc.com/nextclient/ so one can run in a browser - that would get through locked down places.

I am currently working on getting JUnit XML output files (for C.I. reports etc.). So... Master will change a lot, i guess.

[kashperanto]

Hey, I just gave kiwiirc a try, and it appears to be unable to find the tarsio channel on the server (searching brings up nothing). I might try with Chrome, but if it doesn't work on Firefox I'm probably not going to use it ;)

The android client works well while I am able to maintain a signal (I work in a large metal building, so unless I am near my desk I tend not to get anything).

So I pulled the latest changes and it appears to be having problems now due to a lack of pkg-config on my system. Is pkg-config a requirement now? That does not ship with ubuntu, and is also not included in build-essential. I'm not 100% familiar with what pkg-config gives us, but if we can get by without it (perhaps as an option) that would be more flexible.

Make uninstall is also failing due to an invalid expansion to "rm -f-rf ...", which I believe is due to the default value of the implicit variable RM. It expands to "rm -f", so we could eliminate the force flag on all RM uses. I'm thinking that it would also be a good idea to add a space after any of the implicit variables of commands to ensure that flags are properly spaced.

Also, I noticed in the README that you mention man files, but man is unable to find anything for the programs on my system. I don't know if that is in the works, or if something needs to be explicitly done during installation, but I figured I'd mention it here.

I know you're doing a lot of work recently, so I won't try to change anything unless you would like me to :)

[aiobofh]

@kashperanto oh.. I'm so sorry :D I am quite caught up in getting all the wrinkles out of it... Yeah, you're right. I did a (poor) assumption that pkg-config was available (it's pretty much "standard" on Linux systems). I'm so sorry! :( But... I think it makes things easier on Linux at least, there should however be a fall-back to some pre-defined path to find the tools, mk-files, tarsio.c and tarsio.h in that case. E.g. /usr/local/ . What do you think is the best place to install it (and find tarsio.mk when using the shipped tarsio.mk)? I DO very much want it to be as easy to use on as many platforms as possible. I have not encountered a Linux(-like) system without pkg-config tho. If you install it i think you should be able to get going further, or do a manual install and explicitly refer to the paths in your own Makefile or something.

I'm done with the major restructuring (did a shit-load of stuff during the weekend, and held a demo of it). So I will take a look at these things, if you have suggestions and beat me to it, please go a head. I am more than happy if there were more developers/"integrators". :D

[aiobofh]

@kashperanto regarding man-pages: yes. i generate man8-pages in the doc/Makefile (but it requires rst2man to be installed). I will make the installation of man-pages optional. It's easy to read the rst-files directly in the doc-folder tho. I'll try to do a push later today. Both for optional pkg-config and man8-pages. I suppose the install-commands will break in the current master.

I tried KiwIRC at some point... It seemed to work for me - Remember the pad-lock (since it's SSL/TLS encrypted).

Then /join #tarsio

I usually run a text-based IRC client on a SSH + GNU Screen / TMux session. Mosh is popular these days too for generic ssh shell reuse.

[kashperanto]

@aiobofh Hey, no worries, it is fun to see the changes happening in real time :)

I was able to apt install pkg-config ok, and honestly I'm surprised it isn't included by default. I suppose that it is less necessary for most users of Ubuntu due to the ability to use apt to install packages (or the ability to use .deb files when a package is not in the repository). Everything built OK after I installed that.

So the current testing framework I use is Ceedling, and one nice feature about that tool is that (by default) it includes the entire testing environment in the project itself. The idea behind that is to make testing stable over time, so a project can use the version of Ceedling that was used to develop its tests. You can also use the system installation of Ceedling for a project, but the default behavior is to copy the current system installation into the project's folder when making a new project. This is good for not breaking tests on old projects where you would really not want to go back and try to install old versions of tools just to get tests going.

This is definitely not the most efficient solution, but it would be a useful feature to support. It would also solve the "where does tarsio live" problem, and may provide an easier solution to making it work on windows. For the embedded C use case I think being able to work without a system installation on windows would get more acceptance (probably being called from a vendor's eclipse-based IDE). If someone is worried about the small amount of extra space taken up by a C executable and source code they should probably invest in a new hard drive ;)

Rst files are fine by me, and would be a good solution for a project-based installation of tarsio. I have no problem installing whatever packages are needed to help test the system-based install :)

I didn't think to try the /join command manually, but their "channel search" feature was unable to find the tarsio channel. I got into the base server fine, though. I'll see if the text command does the trick.

[aiobofh]

@kashperanto ok, i managed to make a refactoring on how tarsio.mk looks for the Tarsio tools.

I wrote a bit about it in the README.rst. Hopefully it will work better now. There are a few things you can tweak to make it find "itself" :D

[kashperanto]

@aiobofh Thanks, I will give it a try. Everything seems to be working with the pkg-config and your examples. I've got a long meeting coming up where I might have time to work on some tests.

[aiobofh]

@kashperanto hey :) I managed to get checks compile and run native with 5 different C-compilers now :) GCC, Clang, SAS/C, VBCC and VisualC :) (the environment to do regression is quite specific tho, with various emulators/simulators for the more "obscure" ones). I think I'm closing in on a 0.9 tagged release soon enough.

I'm thinking a good thing to try out is to develop something embedded (texas or what not) host-only. Making includes of some weird SDK just to see if the function prototypes are parsed, and all datatypes snooped up so that unit-checks can be automated in a world where one simply swap out the compiler to one as fast as possible on the host. I think it's much up to the quality/portability of the SDK header files.

[kashperanto]

@aiobofh Hey, good to hear it is working on so many platforms.

I have an older project that I have been meaning to write some tests for that uses Freescale/NXP Processor Expert, which caused me many headaches when I was trying to use Ceedling. Processor Expert generates code that is not very modular, and in Ceedling I ended up just having to make a test version of their include files to avoid nested inclusions and many other problematic practices.

I can try it out with some code that uses TI's TivaWare drivers for a more humane test. They are more reflective of where things are headed (at least I hope so).

[aiobofh]

@kashperanto i need to collect all typedefs and other datatype definitions from "a foreign SDK" and generate it into the _data.h file, it seems, to make this smooth. This way all the stuff needed is available for the check-cases with just including the generated header file. (Making includes of the SDK in the check-file just messes things up for the compilers) I was playing around with this in the beginning, but since I mainly focused on host development initially I left it for the future. But I'm making good progress on enabling host-only TDD for an "unknown" target platform and compiler. I hope :D

For now I suggest you just familiarize yourself in host-only environment. :)

[kashperanto]

@aiobofh Hey, sorry for the long delay since my last update. I have attempted to jump straight to the worst-case code that uses Processor Expert (PE for short), and have been hitting a lot of minor issues (some due to my inexperience with make). Today I reached the point where I think my configuration is correct, but I am running into problems with non-standard compiler features that are used in the generated driver code. I am using the "in-repo" tarsio option, which seems to be easy enough with your makefiles :)

For some more background, PE is essentially a plugin for Eclipse that you use to generate driver code for Freescale/NXP microcontrollers. It gives you a GUI that lets you specify what types of peripherals you are using and how you want them to be configured, so in most cases you don't even have to refer to the datasheet to get things up and running.

PE works well enough, but the code they generate is not very modular at all, so it is hard to isolate it for unit testing. They do things like #include a long list of other headers (even if nothing is used in them), and they also map a lot of variables for registers to their absolute addresses (yes, in a header file). For some simpler driver functions PE also generates function-like macros instead of actual functions (but the naming convention does not change). To get around these problems with Ceedling I basically had to strip down several of the PE header files to make dummy versions that contained only the function prototypes. For more modern HAL libraries like TivaWare I have had luck with Ceedling being able to parse them and make the mocks without issue (in most cases).

So one feature suggestion that would be very useful for embedded work would be to add some type of "support" directory that gets added to the start of the path when building test files. I'm currently trying to hack this into my makefiles, but it would be good to have a standard folder where those "override" files can live. This would also be good for non-embedded edge cases. Someone was having problems using Ceedling to test code using glib on linux, because glib.h includes other header files, and the mocking tool does not traverse an include tree. The "override" trick was the solution for their problem as well.

Aside from that I did notice that the TBUILDROOT and TOBJROOT do not appear to work. Make claims that there is no rule to make my check file whenever either of these is defined. I am following the 2_out_of_source_checks/ example, and if I update that to include those folders it also fails. I will have to dig more into the main makefile to understand why this is the case (so I am hoping this makes sense to you ;)

Also, I am wondering if we could put a standard way of including source folders into the makefile? In most of my projects the source will be separated into at least two locations: one for the source of the project itself, and one for the board support package. We usually have one source folder, one BSP folder, and one for generated code (or for the hardware abstraction layer). The source folder is usually at least a few folders deep, too. Our typical compiler argument list is full of "-I". For this test project I just added my required folders to the CFLAGS in the test folder's Makefile, but for more complex projects that may be unsustainable.

[aiobofh]

@kashperanto hi there :) Awesome to hear from you, and THANK YOU for nice ideas and thoughts. So, on with the business...

Right... I'm not super-duper focused on making the .mk-file support all conceivable use-cases. At leas not right now. However awesome it would be. The reason is: Most build systems/project directory structures are their own unique universe :D and often written by people that do not understand build systems. This end up with a gazillion of highly specialized GNU Make or CMake systems, all living in the skies of special snow-flakes. With that said it would be cool to be able to do it. I totally agree! I've been down that road before, with "some" success in cutest. Problem is to keep the tarsio.mk useful, readable and clear.

The tarsio.mk file was originally mainly thought of as an example and to be flexible enough to try out some examples. If you take a look at the tarsio_dist.mk in the source-tree you see what happens when you try to be "clever" on supporting the whole world :D It is NOT a sight for sore eyes. :D I already see tarsio.mk closing in on tarsio_dist.mk... For better or for worse.

The TBUILDROOT and TOBJROOT stuff I need to take a look at, probably i will continue the stuff i'm focusing on on a branch, and try to help you out with fixing those. Or if you find a way to make it work, please go a head! It seamed you have thought a bit about it already.

The support-directory stuff sounds interesting! I'm not 100% sure what you mean, but it seems like you want to prepend/replace the inc/src-folder for some files, where you want "stubs" or host-side implementations? (I'll probably run in to this soon enough myself :D )

So... My thinking is: If someone is just able to use the tam, tcg, tmg, tsg and ttg (the more vital part of Tarsio) tools they can be incorporated into an existing build system, manual, or scripted or what not. But probably it could be a completely separate build-system for checks. So "weird" directory structures would be nice to support with tarsio.mk at least. Then it would obviously be awesome slab on an example into the examples-directory of this repo. I think that would add much value. More examples, for more scenarios on how to use the 5 binaries in a flexible good way. And in the future it may (or may not) be possible to add on the generic tarsio.mk file.

I'm currently working on the datatype extractor in tcg, and an example which enables one to write unit-checks on host-side just having access to the header files of a foreign SDK (will not be shipped with Tarsio). I hope that the approach will work better than Ceedling with your PE scenario. There are some potential issues with conflicting datatypes and prototypes in the libc of the host (probably touching at your "support" directory stuff here)... Still thinking a bit about this... Time will tell how well it will work. But as a general idea I think this will be a killer-feature. The work is currently moving a bit slow, due to reasons.

[kashperanto]

@aiobofh Hey, I definitely agree that keeping it simple and understandable is best. The way Ceedling handles the build situation is by just letting you specify any number of folders for source code, test code, and support code; and then it treats them all the same within each group (source/test/support). I believe the only difference between source and support folders is that support is added to the include path earlier when building the test code, so duplicate files will be found there first. The test/ folder is required to be an exact mirror of the source/ folder, so for source/some_folder/some_file.c you would have test/some_folder/test_some_file.c. This design has worked for all of the project structures I have seen here, but I'm sure it would break for someone.

I will try to figure out the TBUILDROOT/TOBJROOT stuff once I get an actual test running. I am currently hitting some type definition problems, but it is probably an include issue somewhere that I need to sort out.

It is just a nice convenience to have this practice built into the tool, but it is by no means required to make things work. Sometimes trying to make things convenient backfires by hiding too much information from the user. At the same time, I know plenty of people who will not use something if they have to constantly touch a makefile. I intend to level up my makefiling skills, so maybe I will be able to put something together that works for the typical embedded project without requiring much knowledge of make. Maybe if we get put into quarantine for coronavirus I will have more time to work on this ;)

The support directory thing is really just a method for organizing the include path provided to gcc, so nothing too fancy here. The support directories just get added to the start of the list.

I like the idea of having some good examples for how to use the binaries in flexible/good ways first.

So does the current tarsio implementation require the c code even when you don't use the module-checks? In Ceedling I only need to give it the header file, so that is usually all I copy to the support folder when overriding a module. The CMock utility that it uses only requires a header file to generate the mock functions. If that is the case, then maybe my problems are caused by not having the source code for these driver modules...

Maybe the datatype extractor is what I am missing ;)

[kashperanto]

@aiobofh Hey, so I just managed to get my first test to build after adding in some dummy .c files for my dummy headers. I also had to include my dependency (file included in the .c file for the UUT) in the _check.c file, for things to work properly. It is working now, though :)

Before I did this I was getting some very weird errors from gcc that pointed to obviously incorrect locations:

/mnt/c/Users/me/Projects/test_project/ $ make check Q=
make --no-print-directory -C tests
mkdir -p ./ && touch .placeholder
cc -Wall -pedantic -std=c99 -I. -I../support/  -I../tarsio/inc -I../../Sources/  -o tarsio.o -c ../tarsio/src/tarsio.c
cc -O0 -Wall -pedantic -std=c99 -I. -I../support/  -I../tarsio/inc -I../../Sources/ -Dmain=__tarsio_replace_main -E -c ../../Sources/nvm.c > nvm.pp
/usr/local/bin/tcg nvm.pp nvm.sym
/usr/local/bin/tsg nvm.sym nvm_check.c > nvm_data.h
/usr/local/bin/tam nvm.sym nvm.pp nvm_data.h > nvm_proxified.pp
cc -Wall -pedantic -std=c99 -I. -I../support/  -I../tarsio/inc -I../../Sources/ -x cpp-output -c nvm_proxified.pp -o nvm_proxified.o
../../Sources/nvm.c:105:27: error: invalid flag "" in line directive
     // Only allow writes if page is empty or good
                           ^
../../Sources/nvm.c:105:28: warning: extra tokens at end of ## directive
     // Only allow writes if page is empty or good
                            ^
../../Sources/nvm.c:91:28: error: invalid flag "" in line directive
         Page->NumRecords = 0;
                            ^
../../Sources/nvm.c:91:29: warning: extra tokens at end of ## directive
         Page->NumRecords = 0;
                             ^
../../Sources/nvm.c:139:28: error: invalid flag "" in line directive
     byte u8Errors;
                            ^
../../Sources/nvm.c:139:29: warning: extra tokens at end of ## directive
     byte u8Errors;
                             ^

I am not quite sure what to make of this, but it is pointing to completely benign lines and columns, so I think something must be going on in the generation of the proxified file if I don't have any .c files for the to-be-mocked files.

I might have some time tomorrow to start making some actual tests, so that is a victory :)

If you get something workable with the datatype extractor let me know and I can run it on my nasty Processor Expert files to see what happens ;)

Also, I just noticed that I am using the installed binaries even though I am using the local tarsio method, so that might also need updated (or perhaps I am not setting up the makefile properly).

edit: I have only one check for a single function in the UUT that makes only one call into the driver module:

#include <stdio.h>
#include <unistd.h>
#include <tarsio.h>
#include "IFsh1.h"     // This is the dependency I had to include
#include "nvm.h"      // The UUT, which #includes the above header in its .c file
#include "nvm_data.h"

 check(ErasePage_erases_page_0_with_correct_address) {
   ErasePage(PAGE0);
   assert_eq(1, tarsio_mock.IFsh1_EraseSector.call_count);
   assert_eq(0xE000, tarsio_mock.IFsh1_EraseSector.args.arg0);
 }

[aiobofh]

Hm. I think it's the C++ style comments, if I were to guess. :) There is a TODO on that.