CPU1 not released after boot

[nuraalkhateeb]

Hi Sahand,

I'm now trying to use CPU1 on the HPS side and I was looking through your boot messages that show

[ 0.033308] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001 [ 0.033369] Brought up 2 CPUs [ 0.033384] SMP: Total of 2 processors activated (400.00 BogoMIPS).

I do not see these messages on my end. My setup and system is only bringing up 1 CPU. Would you have any ideas why this is the case? Should I be setting something for CPU1 to be enabled?

[sahandKashani]

It depends what your system is running. If you are running a baremetal program, then you'll need to bring up the 2nd CPU from reset yourself (I have never tried yet, so I can't tell how it is done).

If you are using linux, the default configuration when compiling the kernel (in the tutorial) enables SMP, so the kernel should automatically release the 2nd CPU from reset while it boots up. If I recall correctly, you were using a standard terasic linux image, so we don't know how the kernel was configured (they do not say how they create their kernel). It is possible they did not enable that feature.

[nuraalkhateeb]

I'm not running a baremetal program.

I changed back to using the linux image you have in the tutorial a while ago to perform different tests and yet the boot messages don't say that cpu1 is being released. I'll look into SMP though and double check what I have on the sdcard. Let me know if you think of anything else.

Thank you Sahand!

[nuraalkhateeb]

DE0_NANO_boot_messages.txt

These are the messages I'm currently getting.

[sahandKashani]

There is something weird in your output. The FPGA fails to configure at boot time.

reading socfpga.rbf
** Unable to read file socfpga.rbf **
Wrong parameters for FPGA request
fpga - loadable FPGA image support

What board are you using? I see DE0-Nano as the user at the very bottom of the text file, but when I look at the first few lines of the kernel boot images, I see this:

Machine model: Altera SOCFPGA Cyclone V

Whereas I have the following (for the DE0-Nano-SoC board)

Machine model: Terasic DE-0(Atlas)

Are you using the correct device tree? There is a point in the tutorial where you have to compile a specific device tree which corresponds to the board you are using. I'm under the impression you are somehow using the wrong one.

Also, there seems to be pieces of the terasic rootfs on your sdcard

cat: /sys/class/fpga/fpga0/status: No such file or directory
/etc/rc5.d/S90gsrd_init.sh: 5: [: unexpected operator
Starting blinking LED server
Stopping Bootlog daemon: bootlogd.

Maybe try to format the whole sdcard and rewrite everything to it instead of only changing specific pieces. I'm under the impression the sdcard is badly configured.

[nuraalkhateeb]

I'm programming the FPGA through JTAG at the moment until the design is final. Then I'll convert the sof to rbf. Could that be a problem?

[nuraalkhateeb]

I believe I have the correct device tree but I'll double check that as well.

[sahandKashani]

I think some drivers may crash when the kernel is booting if they try to access FPGA resources which are unavailable. However, this shouldn't affect the bringup of the HPS (at least, I don't see why it should affect it).

[nuraalkhateeb]

Thanks Sahand. I did convert my .sof to .rbf and put it on my sd card.

I was looking at the device tree that was loaded to the sd card and

   cpus {
    #address-cells = <0x1>;
    #size-cells = <0x0>;
    enable-method = "altr,socfpga-smp";

    cpu@0 {
        compatible = "arm,cortex-a9";
        device_type = "cpu";
        reg = <0x0>;
        next-level-cache = <0x1>;
    };

    cpu@1 {
        compatible = "arm,cortex-a9";
        device_type = "cpu";
        reg = <0x1>;
        next-level-cache = <0x1>;
    };
};

the cpus were not defined.

So I did update the .dtb on the sdcard and SMP started activating both processors.

[nuraalkhateeb]

Sahand,

Were you able to verify that CPU1 was being used?

[sahandKashani]

Yes, both CPUs are used on my device.

I'm just wondering where you got your device tree from, as it sounds weird that the CPUs would not be defined. I use the following one (untouched) in the tutorial: /arch/arm/boot/dts/socfpga_cyclone5_de0_sockit.dtb. Are you using the same one?

[nuraalkhateeb]

How are you verifying that both CPUs are being used?

I'm worried that I was using the wrong device tree. So instead of using socfpga_cyclone5_de0_sockit, I may have been using socfpga_cyclone5_sockit. Therefore, I was not seeing the 2 CPUs activated?! I'll have to look into that more when I get a chance.

[sahandKashani]

If I boot, I see the following:

Brought up 2 CPUs
SMP: Total of 2 processors activated (400.00 BogoMIPS).

Also, if in linux I cat /proc/cpuinfo, I see 2 processors listed. Same when I launch top or htop once the OS has loaded.

[nuraalkhateeb]

Sorry, I meant if you were specifying certain tasks to be done on CPU0 and others on CPU1 and how you were verifying if that was done.

Thank you Sahand.

[sahandKashani]

Technically, you can't directly specify that a task must be executed on a particular CPU when an operating system is running. (Well, technically you can if you set the CPU affinity of a program. I know Windows has one setting, but I don't know if Linux does). The OS scheduler is in charge of assigning a task to a CPU.

I have written multi-threaded applications for these devices before, but that doesn't guarantee that it isn't the same CPU executing the 2 threads in sequence. However, I believe it is the case if the OS knows there are 2 CPUS in /proc/cpuinfo. Probably the best way to know is to find a way to print the CPU identifier in a multi-threaded application to be certain.

[nuraalkhateeb]

I was also testing with threads and found that sometimes the same CPU will call both and other times the OS would switch between the CPUs.

I did find this on https://rocketboards.org/foswiki/view/Documentation/PreloaderUbootCustomization131

Changing Linux command line arguments to isolate an HPS core via U-Boot

U-Boot can pass boot arguments to the Linux kernel to achieve different goals. It is sometimes necessary to keep Linux from scheduling processes in a given CPU so that that CPU could be dedicated to a given application. This is achieved by first telling Linux to isolate the CPU core we wish to dedicate to our application, and then having our application indicate that it wishes to run on the CPU that has been isolated. To isolate the CPU in a multi core system, we change the boot arguments U-Boot passes to Linux and recompile U-Boot. In the Altera Cyclone V board this is accomplished by modifying the file: include/configs/socfpga_common.h by changing the line:

define CONFIG_BOOTARGS "console=ttyS0," __stringify(CONFIG_BAUDRATE)

to

define CONFIG_BOOTARGS "isolcpus=1 console=ttyS0," __stringify(CONFIG_BAUDRATE)

In this example the second CPU of the processor is being isolated. (CPU numbering is zero based) Now, just compile and install U-Boot into your board. When Linux boots, you could verify the command line arguments passed to it by typing: cat /proc/cmdline You can also run htop to view the CPU assigned to each running process. They should all say 1. (htop uses one based CPU numbering). Note: If the command line arguments are not what's expected, try rebooting and defaulting the board's U-Boot environment by doing the following: Interrupt the boot sequence and at the U-Boot command prompt enter the following two commands and reboot: env default -a saveenv The second part of the process involves changing your application to run in the CPU core that has been isolated. Somewhere in your application initialization code do this: unsigned int cpu = 1 // zero based index of the CPU being targeted cpu_set_t mask; CPU_ZERO(&mask); CPU_SET(cpu, &mask); assert(sched_setaffinity(0, sizeof(mask), &mask) == 0); Now run your application and run htop again. You should see that only your application is running on the second CPU.

[nuraalkhateeb]

I haven't tested this yet but do plan on doing so.

[sahandKashani]

Are you sure you need to isolate the CPU for your application? Is your application so timing critical that you cannot tolerate intermittent context switches that the OS does? CPUs are very fast, especially for control-related operations, so I'm not sure if you would get any benefit from isolating a specific CPU. Maybe you could try the simple non-isolated design and see how it goes, and if performance is not satisfactory, then look into the isolcpu option. Having never tested this sort of thing, one cannot know of any eventual problems that arise when it is enabled.

[nuraalkhateeb]

I'm working with an engineer who's application is time critical but again we may not find any gain to using this method. Like you so said we may come across more problems, however, we'll probably still test this method and see the results.

Thanks Sahand.