8sa1-binutils-gdb/libctf/testsuite/lib/ctf-lib.exp
Nick Alcock 35a01a0454 libctf, ld: fix symtypetab and var section population under ld -r
The variable section in a CTF dict is meant to contain the types of
variables that do not appear in the symbol table (mostly file-scope
static declarations).  We implement this by having the compiler emit
all potential data symbols into both sections, then delete those
symbols from the variable section that correspond to data symbols the
linker has reported.

Unfortunately, the check for this in ctf_serialize is wrong: rather than
checking the set of linker-reported symbols, we check the set of names
in the data object symtypetab section: if the linker has reported no
symbols at all (usually if ld -r has been run, or if a non-linker
program that does not use symbol tables is calling ctf_link) this will
include every single symbol, emptying the variable section completely.

Worse, when ld -r is in use, we want to force writeout of every
symtypetab entry on the inputs, in an indexed section, whether or not
the linker has reported them, since this isn't a final link yet and the
symbol table is not finalized (and may grow more symbols than the linker
has yet reported).  But the check for this is flawed too: we were
relying on ctf_link_shuffle_syms not having been called if no symbols
exist, but that function is *always* called by ld even when ld -r is in
use: ctf_link_add_linker_symbol is the one that's not called when there
are no symbols.

We clearly need to rethink this.  Using the emptiness of the set of
reported symbols as a test for ld -r is just ugly: the linker already
knows if ld -r is underway and can just tell us.  So add a new linker
flag CTF_LINK_NO_FILTER_REPORTED_SYMS that is set to stop the linker
filtering the symbols in the symtypetab sections using the set that the
linker has reported: use the presence or absence of this flag to
determine whether to emit unindexed symtabs: we only remove entries from
the variable section when filtering symbols, and we only remove them if
they are in the reported symbol set, fixing the case where no symbols
are reported by the linker at all.

(The negative sense of the new CTF_LINK flag is intentional: the common
case, both for ld and for simple tools that want to do a ctf_link with
no ELF symbol table in sight, is probably to filter out symbols that no
linker has reported: i.e., for the simple tools, all of them.)

There's another wrinkle, though.  It is quite possible for a non-linker
to add symbols to a dict via ctf_add_*_sym and then write it out via the
ctf_write APIs: perhaps it's preparing a dict for a later linker
invocation.  Right now this would not lead to anything terribly
meaningful happening: ctf_serialize just assumes it was called via
ctf_link if symbols are present.  So add an (internal-to-libctf) flag
that indicates that a writeout is happening via ctf_link_write, and set
it there (propagating it to child dicts as needed).  ctf_serialize can
then spot when it is not being called by a linker, and arrange to always
write out an indexed, sorted symtypetab for fastest possible future
symbol lookup by name in that case.  (The writeouts done by ld -r are
unsorted, because the only thing likely to use those symtabs is the
linker, which doesn't benefit from symtypetab sorting.)

Tests added for all three linking cases (ld -r, ld -shared, ld), with a
bit of testsuite framework enhancement to stop it unconditionally
linking the CTF to be checked by the lookup program with -shared, so
tests can now examine CTF linked with -r or indeed with no flags at all,
though the output filename is still foo.so even in this case.

Another test added for the non-linker case that endeavours to determine
whether the symtypetab is sorted by examining the order of entries
returned from ctf_symbol_next: nobody outside libctf should rely on
this ordering, but this test is not outside libctf :)

include/ChangeLog
2021-01-26  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-api.h (CTF_LINK_NO_FILTER_REPORTED_SYMS): New.

ld/ChangeLog
2021-01-26  Nick Alcock  <nick.alcock@oracle.com>

	* ldlang.c (lang_merge_ctf): Set CTF_LINK_NO_FILTER_REPORTED_SYMS
	when appropriate.

libctf/ChangeLog
2021-01-27  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-impl.c (_libctf_nonnull_): Add parameters.
	(LCTF_LINKING): New flag.
	(ctf_dict_t) <ctf_link_flags>: Mention it.
	* ctf-link.c (ctf_link): Keep LCTF_LINKING set across call.
	(ctf_write): Likewise, including in child dictionaries.
	(ctf_link_shuffle_syms): Make sure ctf_dynsyms is NULL if there
	are no reported symbols.
	* ctf-create.c (symtypetab_delete_nonstatic_vars): Make sure
	the variable has been reported as a symbol by the linker.
	(symtypetab_skippable): Mention relationship between SYMFP and the
	flags.
	(symtypetab_density): Adjust nonnullity.  Exit early if no symbols
	were reported and force-indexing is off (i.e., we are doing a
	final link).
	(ctf_serialize): Handle the !LCTF_LINKING case by writing out an
	indexed, sorted symtypetab (and allow SYMFP to be NULL in this
	case).  Turn sorting off if this is a non-final link.  Only delete
	nonstatic vars if we are filtering symbols and the linker has
	reported some.
	* testsuite/libctf-regression/nonstatic-var-section-ld-r*:
	New test of variable and symtypetab section population when
	ld -r is used.
	* testsuite/libctf-regression/nonstatic-var-section-ld-executable.lk:
	Likewise, when ld of an executable is used.
	* testsuite/libctf-regression/nonstatic-var-section-ld.lk:
	Likewise, when ld -shared alone is used.
	* testsuite/libctf-regression/nonstatic-var-section-ld*.c:
	Lookup programs for the above.
	* testsuite/libctf-writable/symtypetab-nonlinker-writeout.*: New
	test, testing survival of symbols across ctf_write paths.
	* testsuite/lib/ctf-lib.exp (run_lookup_test): New option,
	nonshared, suppressing linking of the SOURCE with -shared.
2021-02-04 16:01:53 +00:00

424 lines
12 KiB
Plaintext

# Support routines for libctf testsuite.
# Copyright (C) 1994-2021 Free Software Foundation, Inc.
#
# This file is part of the GNU Binutils.
#
# This file is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street - Fifth Floor, Boston,
# MA 02110-1301, USA.
proc load_common_lib { name } {
global srcdir
load_file $srcdir/../../binutils/testsuite/lib/$name
}
load_common_lib binutils-common.exp
proc run_native_host_cmd { command } {
global link_output
global ld
verbose -log "$command"
set run_output ""
try {
set run_output [exec "sh" "-c" "$command" "2>@1"]
set status 0
} trap CHILDSTATUS {results options} {
set status [lindex [dict get $options -errorcode] 2]
set run_output $results
}
regsub "\n$" $run_output "" run_output
if { [lindex $status 0] != 0 && [string match "" $run_output] } then {
append run_output "child process exited abnormally"
}
if [string match "" $run_output] then {
return ""
}
verbose -log "$run_output"
return "$run_output"
}
proc run_host_cmd { prog command } {
global link_output
global gcc_B_opt
global gcc_ld_B_opt_tested
global ld
if { ![is_remote host] && [which "$prog"] == 0 } then {
perror "$prog does not exist"
return 0
}
# If we are compiling with gcc, we want to add gcc_B_opt to flags. However,
# if $prog already has -B options, which might be the case when running gcc
# out of a build directory, we want our -B options to come first.
set gccexe $prog
set gccparm [string first " " $gccexe]
set gccflags ""
if { $gccparm > 0 } then {
set gccflags [string range $gccexe $gccparm end]
set gccexe [string range $gccexe 0 $gccparm]
set prog $gccexe
}
set gccexe [string replace $gccexe 0 [string last "/" $gccexe] ""]
if {[string match "*cc*" $gccexe] || [string match "*++*" $gccexe]} then {
set gccflags "$gcc_B_opt $gccflags"
if {![info exists gcc_ld_B_opt_tested]} {
set gcc_ld_B_opt_tested 1
set ld_version_message [run_host_cmd "$ld" "--version"]
set gcc_ld_version_message [run_host_cmd "$prog" "$gccflags -Wl,--version"]
if {[string first $ld_version_message $gcc_ld_version_message] < 0} {
perror "************************************************************************"
perror "Your compiler driver ignores -B when choosing ld."
perror "You will not be testing the new ld in many of the following tests."
set gcc_ld_version [run_host_cmd "$prog" "$gccflags --print-prog-name=ld"]
if {![string match "" $gcc_ld_version] && ![string match "ld" $gcc_ld_version]} {
perror "It seems you will be testing $gcc_ld_version instead."
}
perror "************************************************************************"
}
}
}
verbose -log "$prog $gccflags $command"
set status [remote_exec host [concat sh -c [list "$prog $gccflags $command 2>&1"]] "" "/dev/null" "libctf.tmp"]
remote_upload host "libctf.tmp"
set run_output [file_contents "libctf.tmp"]
regsub "\n$" $run_output "" run_output
if { [lindex $status 0] != 0 && [string match "" $run_output] } then {
append run_output "child process exited abnormally"
}
remote_file build delete libctf.tmp
remote_file host delete libctf.tmp
if [string match "" $run_output] then {
return ""
}
verbose -log "$run_output"
return "$run_output"
}
proc run_host_cmd_yesno { prog command } {
global exec_output
global errcnt warncnt
set exec_output [prune_warnings [run_host_cmd "$prog" "$command"]]
# Ignore error and warning.
set errcnt 0
set warncnt 0
if [string match "" $exec_output] then {
return 1;
}
return 0;
}
# Return true if we can build a program with the compiler.
# On some targets, CC might be defined, but libraries and startup
# code might be missing or require special options that the ld test
# harness doesn't know about.
proc check_compiler_available { } {
global compiler_available_saved
global CC
if {![info exists compiler_available_saved]} {
if { [which $CC] == 0 } {
set compiler_available_saved 0
return 0
}
set flags ""
if [board_info [target_info name] exists cflags] {
append flags " [board_info [target_info name] cflags]"
}
if [board_info [target_info name] exists ldflags] {
append flags " [board_info [target_info name] ldflags]"
}
set basename "tmpdir/compiler[pid]"
set src ${basename}.c
set output ${basename}.out
set f [open $src "w"]
puts $f "int main (void)"
puts $f "{"
puts $f " return 0; "
puts $f "}"
close $f
if [is_remote host] {
set src [remote_download host $src]
}
set compiler_available_saved [run_host_cmd_yesno "$CC" "$flags $src -o $output"]
remote_file host delete $src
remote_file host delete $output
file delete $src
}
return $compiler_available_saved
}
# Compile and link a C source file for execution on the host.
proc compile_link_one_host_cc { src output additional_args } {
global CC_FOR_HOST
global CFLAGS
return [run_native_host_cmd "./libtool --quiet --tag=CC --mode=link $CC_FOR_HOST $CFLAGS $src -o $output $additional_args" ]
}
# Compile a C source file, with the specified additional_flags.
proc compile_one_cc { src output additional_flags } {
global CC
global CFLAGS
set flags ""
if [board_info [target_info name] exists cflags] {
append flags " [board_info [target_info name] cflags]"
}
if [board_info [target_info name] exists ldflags] {
append flags " [board_info [target_info name] ldflags]"
}
if [is_remote host] {
set src [remote_download host $src]
}
return [run_host_cmd "$CC" "$flags $CFLAGS $additional_flags $src -o $output"]
}
# run_lookup_test FILE
#
# Compile with the host compiler and link a .c file into a "lookup" binary, then
# compile and optionally link together a bunch of .s or .c files with CTF info
# and pass the name of the resulting binary to the "lookup" binary and check the
# output. (If none is specified, the binary is expected to generate its own CTF
# for testing purposes.)
#
# As with run_dump_test, this is all driven by a file (in this case, a .lk file)
# beginning with zero or more option lines, which specify the names of the
# lookup binary's source file, the source file(s) with CTF info to compile
# together, and whether to link them. The optional lines have the syntax:
#
# # OPTION: VALUE
#
# OPTION is the name of some option, like "name" or "lookup", and
# VALUE is OPTION's value. The valid options are described below.
# Whitespace is ignored everywhere, except within VALUE. The option
# list ends with the first line that doesn't match the above syntax.
# However, a line within the options that begins with a #, but doesn't
# have a recognizable option name followed by a colon, is considered a
# comment and entirely ignored.
#
# The interesting options are:
#
# name: TEST-NAME
# The name of this test, passed to DejaGNU's `pass' and `fail'
# commands. If omitted, this defaults to FILE, the root of the
# lookup .c file's name.
#
# lookup: SOURCE
# Compile the file SOURCE.c. If omitted, the lookup source defaults
# to FILE.c.
#
# source: SOURCE
# Assemble the file SOURCE.c and pass it to the LOOKUP program.
#
# nonshared:
# If set, do not link with -shared.
#
# link:
# If set, link the SOURCE together even if only one file is specified.
#
# link_flags:
# If set, extra flags to pass to the linker.
#
# xfail: GLOB|PROC ...
# This test is expected to fail on a specified list of targets.
#
# Each option may occur at most once unless otherwise mentioned.
#
# After the option lines come regexp lines. run_lookup_test calls
# regexp_diff to compare the output of the lookup program against the
# regexps in FILE.d.
#
proc run_lookup_test { name } {
global CC CFLAGS LIBS
global copyfile env runtests srcdir subdir verbose
if ![runtest_file_p $runtests $name] then {
return
}
if [string match "*/*" $name] {
set file $name
set name [file tail $name]
} else {
set file "$srcdir/$subdir/$name"
}
set opt_array [slurp_options "${file}.lk"]
if { $opt_array == -1 } {
perror "error reading options from $file.lk"
unresolved $subdir/$name
return
}
set run_ld 0
set shared "-shared"
set opts(link) {}
set opts(link_flags) {}
set opts(nonshared) {}
set opts(lookup) {}
set opts(name) {}
set opts(source) {}
set opts(xfail) {}
foreach i $opt_array {
set opt_name [lindex $i 0]
set opt_val [lindex $i 1]
if { $opt_name == "" } {
set in_extra 1
continue
}
if ![info exists opts($opt_name)] {
perror "unknown option $opt_name in file $file.lk"
unresolved $subdir/$name
return
}
set opts($opt_name) [concat $opts($opt_name) $opt_val]
}
if { [llength $opts(lookup)] == 0 } {
set opts(lookup) "$file.c"
} else {
set opts(lookup) "[file dirname $file]/$opts(lookup)"
}
if { [llength $opts(name)] == 0 } {
set opts(name) $opts(lookup)
}
if { [llength $opts(link)] != 0
|| [llength $opts(source)] > 1 } {
set run_ld 1
}
if { [llength $opts(nonshared)] != 0 } {
set shared ""
}
set testname $opts(name)
if { $opts(name) == "" } {
set testname "$subdir/$name"
}
# Compile and link the lookup program.
set comp_output [compile_link_one_host_cc $opts(lookup) "tmpdir/lookup" "libctf.la"]
if { $comp_output != ""} {
send_log "compilation of lookup program $opts(lookup) failed with <$comp_output>"
perror "compilation of lookup program $opts(lookup) failed"
fail $testname
return 0
}
# Compile the inputs and posibly link them together.
set lookup_output ""
if { [llength $opts(source)] > 0 } {
set lookup_flags ""
if { $run_ld } {
set lookup_output "tmpdir/out.so"
set lookup_flags "-gt -fPIC $shared $opts(link_flags)"
} else {
set lookup_output "tmpdir/out.o"
set lookup_flags "-gt -fPIC -c"
}
if [board_info [target_info name] exists cflags] {
append lookup_flags " [board_info [target_info name] cflags]"
}
if [board_info [target_info name] exists ldflags] {
append lookup_flags " [board_info [target_info name] ldflags]"
}
set src {}
foreach sfile $opts(source) {
if [is_remote host] {
lappend src [remote_download host [file join [file dirname $file] $sfile]]
} else {
lappend src [file join [file dirname $file] $sfile]
}
}
set comp_output [run_host_cmd "$CC" "$CFLAGS $lookup_flags [concat $src] -o $lookup_output"]
if { $comp_output != ""} {
send_log "compilation of CTF program [concat $src] failed with <$comp_output>"
fail $testname
return 0
}
}
# Time to setup xfailures.
foreach targ $opts(xfail) {
if [match_target $targ] {
setup_xfail "*-*-*"
break
}
}
# Invoke the lookup program on the outputs.
set results [run_host_cmd tmpdir/lookup $lookup_output]
set f [open "tmpdir/lookup.out" "w"]
puts $f $results
close $f
if { [regexp_diff "tmpdir/lookup.out" "${file}.lk"] } then {
fail $testname
if { $verbose == 2 } then { verbose "output is [file_contents tmpdir/lookup.out]" 2 }
return 0
}
pass $testname
return 0
}
# Returns true if the target compiler supports -gt
proc check_ctf_available { } {
global ctf_available_saved
if {![info exists ctf_available_saved]} {
if { ![check_compiler_available] } {
set ctf_available_saved 0
} else {
set basename "tmpdir/ctf_available[pid]"
set src ${basename}.c
set output ${basename}.o
set f [open $src "w"]
puts $f "int main() { return 0; }"
close $f
set comp_output [compile_one_cc $src $output "-gt -c"]
if { $comp_output == ""} {
set ctf_available_saved 1
} else {
set ctf_available_saved 0
}
remote_file host delete $src
remote_file host delete $output
file delete $src
}
}
return $ctf_available_saved
}