prebuilt-elf-files for test

unicornx commented 2 years ago

the folder bionic/tests/prebuilt-elf-files/riscv64 is created for test and the binaries under this folder are created for testing.

libtest_empty.so is generated by "libtest_empty" in bionic/tests/libs/Android.bp, while others should be created by manually based on the libtest_empty.so. You can compare the other ARCH, such as arm and etc, and refer to https://lists.riscv.org/g/sig-android/message/24.

So please continue,

investigate if AOSP provides scripts on creating the other binaries.
if not, investigate and create them by manual and run test to verify it.

Provide a report when you finish this work.

CambridgeMjq commented 2 years ago

Update 1: School Final week, working on background info research

CambridgeMjq commented 2 years ago

Update2: Working on learing ELF file structure and info, schedule to use the file existed inside arm64 folder to make the new file

unicornx commented 2 years ago

Update2: Working on learing ELF file structure and info, schedule to use the file existed inside arm64 folder to make the new file

You should NOT make new files based on the files inside arm64 folder. Please read the top message on this page. I quote here for your reference：

libtest_empty.so is generated by "libtest_empty" in bionic/tests/libs/Android.bp, while others should be created by manually based on the libtest_empty.so. You can compare the other ARCH, such as arm and etc, and refer to https://lists.riscv.org/g/sig-android/message/24.

CambridgeMjq commented 2 years ago

Update 2.1 & 3: Correct the update2, no change will made in arm folder. I will take arm64 as reference, so that make change in RISCV64. Almost done with research ELF file structure. Access .so file by vscode hex editor, successfully compare libtest_empty.so and libtest_inbalid-empty_shdr_table.so in arm64 folder. Starting working on file in riscv64.

unicornx commented 2 years ago

Update 2.1 & 3: Correct the update2, no change will made in arm folder. I will take arm64 as reference, so that make change in RISCV64. Almost done with research ELF file structure. Access .so file by vscode hex editor, successfully compare libtest_empty.so and libtest_inbalid-empty_shdr_table.so in arm64 folder. Starting working on file in riscv64.

Suggest you give a introduction aside with bin change, I would like to know for each file such as:

what's the reason for change, this may require you to investigate what's the so is created for what test purpose
what will be changed
how to make the change

CambridgeMjq commented 2 years ago

Update 3.2: Use readlf command to read file information and compare libtest_empty.so and libtest empty shdr table to find some different information. Just take these two files as an example. This location stores the number of section headers. Now I want to create an invalid elf with a section header number of 0, so I changed this byte to 0. After modification, readlf confirms that the number of section headers in this elf file has indeed become 0. (The above description is a local operation, the content has not been submitted to the server yet, and will be submitted after further confirmation)

The difficulty part is more than expected, and the recent flight cancellation requires move to different city, which has caused the progress to be relatively slow. It is expected that there will be a certain stable output starting from January 5th, Western time in the United States. Sorry for the delay.

CambridgeMjq commented 2 years ago

Upd 3.3: Convert all .so files in ARM64/riscv64 folder to .hex. Modify some of the .hex files in the riscv64 folder.

unicornx commented 2 years ago

https://zhuanlan.zhihu.com/p/463369525

unicornx commented 2 years ago

I took a look at your analysis report, and I feel that it is not deep enough. Please continue. My suggestion is to carefully study what the test cases corresponding to these binaries are testing, and start from understanding the test requirements to understand why we created these prebuilt so and what to change and how to change.

Copied my previous comments as your reference:

CambridgeMjq commented 2 years ago

Upd 4: https://zhuanlan.zhihu.com/p/470452698 case analysis and modify plan

unicornx commented 2 years ago

The 9 files under bionic/tests/prebuilt-elf-files/riscv64 are used for dlfcn testcases. Check bionic/tests/dlfcn_test.cpp for details descriptions: Corresponding relationship is listed below:

dlfcn.dlopen_invalid_rw_load_segment <-------- libtest_invalid-rw_load_segment.so
dlfcn.dlopen_invalid_unaligned_shdr_offset <-------- libtest_invalid-unaligned_shdr_offset.so
dlfcn.dlopen_invalid_zero_shentsize <-------- libtest_invalid-zero_shentsize.so
dlfcn.dlopen_invalid_zero_shstrndx <-------- libtest_invalid-zero_shstrndx.so
dlfcn.dlopen_invalid_empty_shdr_table <-------- libtest_invalid-empty_shdr_table.so
dlfcn.dlopen_invalid_zero_shdr_table_offset <-------- libtest_invalid-zero_shdr_table_offset.so
dlfcn.dlopen_invalid_zero_shdr_table_content <-------- libtest_invalid-zero_shdr_table_content.so
dlfcn.dlopen_invalid_textrels <-------- libtest_invalid-textrels.so
dlfcn.dlopen_invalid_textrels2 <-------- libtest_invalid-textrels2.so

After you change, you can run bionic-unit-tests to verify it.

Details about how to run unit tests, please read https://github.com/aosp-riscv/test-riscv/blob/riscv64-android-12.0.0_dev/docs/howto-setup-test-env.md.

CambridgeMjq commented 2 years ago

Update5: When I check the file libtest_invalid-textrels.so, I can Currently only one character difference is found in binary files. But can't understand the problem that this byte points to

unicornx commented 2 years ago

Update5: When I check the file libtest_invalid-textrels.so, I can Currently only one character difference is found in binary files. But can't understand the problem that this byte points to

Please note you can also look at which field this place corresponds to the elf format. And which ARCH are you comparing, I suggest you first look at other ARCHs except riscv.

You can read the place where "has text relocations" is output in bionic/linker/linker.cpp, see under what circumstances in the code will print these error string and then infer how to construct the binary for such cases. This method is also applied on "dlopen_invalid_textrels2".

CambridgeMjq commented 2 years ago

Update 5.1 progress upd, accoding to linker.cpp, and some other expansion files, it is known that shared objects must not contain text relocations starting from Android 6. Therefore, the invalid_textrels/invalid_textrels2 file is to test whether the file has two possibilities for text relocation: the entry/flag part. Explore further how to modify dynamic sections.

CambridgeMjq commented 2 years ago

Upd 5.2 Done with invalid-zero shstrndx / shentsize. Shentsize is about to test size in bytes of the section header. A section header is one entry in the section header table; all entries are the same size. Shstrndx is about test the section header table index of the entry associated with the section name string table.

Project remaining: libtest_invalid-zero_content/offset. (2 files) The former is locating the position and trying to modify it manually, the latter is having difficulty locating the offset. Although the principle is clear for Textrels, there are still some difficulties in modifying the dynamic header part.

Application: Can I push the five files that have been completed?

CambridgeMjq commented 2 years ago

Upd 5.2.1 : Libtest_invalid-zero_shdr_offset The most troublesome point for this file is that no matter how to use the readelf tool, you can't locate the offset position and why you choose that amount

unicornx commented 2 years ago

Upd 5.2 Done with invalid-zero shstrndx / shentsize. Shentsize is about to test size in bytes of the section header. A section header is one entry in the section header table; all entries are the same size. Shstrndx is about test the section header table index of the entry associated with the section name string table.

Project remaining: libtest_invalid-zero_content/offset. (2 files) The former is locating the position and trying to modify it manually, the latter is having difficulty locating the offset. Although the principle is clear for Textrels, there are still some difficulties in modifying the dynamic header part.

Application: Can I push the five files that have been completed?

I would suggest you complete all binaries and PR in whole. Please make sure related test-cases are passed as part of your PR Note: please update your tech report as per my comments before (copied here below for your quick reference). I would like to review the details before merging.

I would like to know for each file such as:

what's the reason for change, this may require you to investigate what's the so is created for what test purpose

what will be changed

how to make the change

CambridgeMjq commented 2 years ago

Update 6:

Update 6.1: Modifications to the file libtest_invalid-zero_shdr_table_content.so

This file is used to test whether the directory of the section header is complete
Modification content: By parsing the file, we can know that there are 19 tables in the Section header part of the file, and each table has ten modules: name, type, flags, address, offset, size, link, info, addralign ,entsize.
Modify the content to clear these ten modules of all tables. A modified version of this file has been uploaded to my directory.

Update 6.2: At present, after reading the detailed information of libtest_invalid-zero_shdr_offset through the readelf tool and related instructions, the comparison results of libtest empty and the previous conclusions are somewhat different, and further exploration is underway.

CambridgeMjq commented 2 years ago

Upd 7: tech report with 5 files https://zhuanlan.zhihu.com/p/491377214

unicornx commented 2 years ago

this issue lasts too long and not resolved. close it now

aosp-riscv / platform_bionic

prebuilt-elf-files for test #2