Amos’ Blog

C linker symbols

2022-01-03T14:00:00+00:00

Disclaimer: this post is a copy of an answer found on the VIS Community Solution platform. The answer was so well written that I felt it was a shame not to share it with as many people as possible. You can find the original answer here, full credits are given to Theo Weidmann (weidmann@ethz.ch).

Origin

This answer originated from a discussion between students. The discussion was caused by a question which read as follows:

For each listed identifier in the following C object file, say whether it is a strong linker symbol, a weak linker symbol, a macro, a symbol local to the compilation unit, or none of these. The C program contained, together wiht many other stuff, the declaration (but not definition) of a function, the declaration of a extern char * and the definition of a struct. All three of those were said not to belong to any of the above possibilities, and this generated a lot of confusion.

What is a symbol?

A symbol is a name for a memory address within assembly code.

Let’s look at an example, test.s. In the below code doSomething and theCount and doAnotherThing are symbols:

  .globl  theCount
  .bss
  .size   theCount, 4
theCount:

  .zero   4
  .text
  .globl  doSomething
doSomething:
  pushq   %rbp
  movq    %rsp, %rbp
  movl    $0, %eax
  call    doAnotherThing
  popq    %rbp
  ret

Note that theCount and doSomething appear as labels, i. e. they appear at the beginning of a line followed by a colon, which defines the symbol. By contrast, doAnotherThing only appears as an instruction operand. We call a symbol that appears as an operand a reference.

So test.s does define theCount and doSomething but it does not define doAnotherThing.

Let’s assemble our test file into an object file:

gcc test.s -c

The assembler creates a symbol table as part of the object file, which is a list of all symbols it encountered. With each symbol, it notes whether the file defined or only referenced the symbol. We can use nm to look at the generated symbol table:

                 U doAnotherThing
0000000000000000 T doSomething
0000000000000000 B theCount

B and T mean that these symbols were defined in the bss and text section, respectively. The U before doAnotherThing means that the symbol is not defined but was referenced.

What are symbols and the symbol table good for?

Linking allows us to split up code. It allows us to define doAnotherThing in one file and use it from another file.

So let’s define doAnotherThing in anotherThing.s:

  .globl  doAnotherThing
doAnotherThing:
  pushq   %rbp
  movq    %rsp, %rbp
  popq    %rbp
  ret

If assembled, the symbol table looks like this:

0000000000000000 T doAnotherThing

So far, we have only created individual object files. To create a binary from these object files, we need to involve the linker.

The linker reads all symbol tables and ensures that for every reference to a symbol, there is a definition of that symbol. In practice, this means checking that all symbols with a U entry in one symbol table have a defining entry in another table from another file.

The actual step of replacing symbol names with addresses has been covered in exercises.

What about strong and weak?

Everything seems good so far. We can use symbols in one file and provide a definition in another file. But it turns out there are cases where we want to define the same symbol in multiple files.

Take, for example, this (purely hypothetical) situation:

We’ve built a huge library of very complex operations. Since most applications only ever use one of these complex operations, every operation (which consists of many functions) is provided as a single object file. Some of these operations need a helper function performCalculation. So we define it in the object file of each operation that needs it. But this leads to an interesting problem when someone links two object files that both contain performCalculation into their program. Which of the two definitions should the linker use?

If we allowed any symbol to be defined more than once, this could have disastrous effects. What if we name two things the same by accident? So by default, the linker will only allow one definition of every symbol. We call symbols that can only be defined one time strong. If a strong symbol is defined multiple times, the linker will give an error.

Still, we want to solve our problem. We do so by introducing an option to allow multiple definitions of the same symbol. We call these symbols weak. To fix our issue, we make performCalculation weak in all our files (note the first line):

  .weak   performCalculation
performCalculation:
  pushq   %rbp
  movq    %rsp, %rbp
  movl    %edi, -20(%rbp)
  movl    %esi, -24(%rbp)

If there are multiple performCalculation definitions at link-time, the linker will choose any of these. Since all implementations of performCalculation are the same, this will work out fine. Problem solved.

The relation between strong and weak was thoroughly covered in the lecture ETH “System Programming and Computer Architecture” course, 251-0061-00L.

At The Level of C

So what does all this mean at the level of C?

First of all, it means that we can only define symbols that were actually defined in C code. If we have

 struct element *pull();

we cannot meaningfully define this symbol. We don’t even know what this function does!

The keyword extern means exactly the same thing:

extern char *otherstring;

It tells the compiler that there is a variable otherstring in one of the other translation units (resulting object files). But this is not the definition. So again, it would not make any sense to emit a symbol definition.

These declarations are only necessary to allow the C compiler to do type checking. (As a matter of fact, early versions of C behaved much like assembly and allowed you to call undeclared, unknown functions…)

This translation unit defines push, head and tail, so they will also result in a symbol definition.

The symbol otherstring will not be referenced in the resulting object file, because we did not use it. If we used it, it would appear as an undefined reference in the symbol table.

Note, that the compiler must always emit definitions as it cannot know which symbols other object files might reference. (The exception being static, of course.)

Final Remarks

But why are uninitialized globals weak? I would guess for historic reasons. The C standard actually prohibits defining variables more than once.

And how to define a weak function in C? You cannot. But other compilers for other languages do emit weak functions.

Where are you taking this from? Weak and strong were not made up by the lecture. It’s widely used. See for example, LLVM, the industry-leading tool for building compilers: https://llvm.org/docs/LangRef.html#linkage-types. (weak and strong is nonetheless an abstraction, the actual Linux implementation is more complicated.)

Deciphering C declaration swith the right-left rule

2021-12-27T13:30:00+00:00

Disclaimer: this post is an adaptation of this blog post, which I felt was not polished enough.

Introduction

Have you ever looked at a C declaration, something like int *x(int *, int) and wondered “Well, what the hell is that?” before proceeding to get an headache trying to understand how to read it? If the answer is yes, then we are in the same boat! Or at least we were, until I discovered this simple rule which makes reading C declarations as easy as it can get.

The Rule

For the following, keep this table in mind:

Symbol(s)	To be read as
`*`	“pointer to”
`[]`	“array of”
`()`	“function returning”

The process is really simple, and it works as follow:

Find the identifier. This is your starting point. Then say to yourself, “identifier is.” You’ve started your declaration.
Look at the symbols on the right of the identifier. If, say, you find () there, then you know that this is the declaration for a function. So you would then have “identifier is function returning”. Or if you found a [] there, you would say “identifier is array of”. Continue right until you run out of symbols OR hit a right parenthesis ). (If you hit a left parenthesis, that’s the beginning of a () symbol, even if there is stuff in between the parentheses. More on that below.)
Look at the symbols to the left of the identifier. If it is not one of our symbols above (say, something like “int”), just say it. Otherwise, translate it into English using that table above. Keep going left until you run out of symbols OR hit a left parenthesis (.

Now repeat steps 2 and 3 until you’ve formed your declaration.

Some Examples

(Taken from the autumun semester exam of the “System Programming and Computer Architecture” course at ETH, 251-0061-00L) Say you want to decipher this not so simple C declaration:

int **x[10]

Start by finding the identifier

int **x[10]
      ^

The declaration reads now as “x is.”. Proceed with step 2 and we get to the end of the line.

int **x[10]
       ^^^^

The declaration is now “x is an array 10 of.”. Proceed with step 3 and we run out of symbols:

int **x[10]
^^^^^^

Our declaration is finally “x is an array 10 of pointer to pointer to int.”. The solution was indeed “declare x as array 10 of pointer to pointer to int”.

Say you now want to decipher the following declaration:

int *x(int *, int)

Start with steps 1 and 2:

int *x(int *, int)
     ^^^^^^^^^^^^^

The declaration is, up until now, “x is a function (pointer to int, int) returning.” Now step 3:

int *x(int *, int)
^^^^^

We obtain “x is a function (pointer to int, int) returning pointer to int.”

The Marching Cubes Algorithm 2: Meshes & Marching Squares

2021-04-10T16:30:00+00:00

Mesh

First steps into meshes

The first thing I had to do was to Google what a mesh is, because I really had no clue. A quick Wikipedia read and I am now a meshes expert! If you are also confused like I was by this term, it turns out being a really simple concept:

In 3D computer graphics and solid modeling, a polygon mesh is a collection of vertices, edges and faces that defines the shape of a polyhedral object. (Courtesy of Wikipedia)

Even though I was trying to accomplish it in 2D, I think you got the idea (and if you didn’t, you wil soon, just keep reading :p): I am going to try to smooth out the borders of the caves from my previous post.

The algorithm

The algorithm is the same as the one we discussed before for the 3D case, but in 2D (that’s why it’s called “Marching Squares”). We start by dividing our space into squares, each one having his 4 vertices on 4 adjacent pixels in the drawing (see Figure 1 for an example).

Figure 1: An example of how the cave is divided into squares, using the seed "Marching Cubes"

Each one of these squares will have 4 control points, one on each one of the square’s edges. These points will then be connected based on which vertices have to be inside the wall and which have to be outside; figure 2 displays all the (16) possible combinations.

Figure 2: All the possible combinations

Figure 3: Close up showing the "control points"

Unfortunately, I had to stop working on this “series” of blog posts because of school reasons :( Hopefully, one day I’ll come back to it.

The Marching Cubes Algorithm 1: Introduction & Procedural Cave Generation in 2D

2021-04-09T16:30:00+00:00

The marching cubes algorithm

Introduction

During the holydays, since ETH wasn’t giving me enough work to do, I decided to learn how to use Unity. While deciding the project to implement, I stumbled upon the marching cube algorithms, so now here we are!

The problem

The main need the Marching Cube Algorithm tries to satisfy, is the need to form a facet approximation to an isosurface through a scalar field, sampled on a rectangle grid. A lot of big words huh? Let’s make this simple: imagine to have a lot of points nicely distributed on a section of space. We now decide which of these points we want to be contained in our 3d surface, and which not. There we are! We can now pass our “marching cube” over all the points and make it draw triangles to “exclude” external vertices: each triangle will have his vertices between an excluded cube vertex and an included one (see Figure 1 for a reference).

The algorithm

The main problem of this algorithm is the amount of data we’d need to compute, but luckily someone has already done that for us, and pre-computed tables are available online. As everyone knows a cube has 8 vertices. Since each one of those vertices can be either inside the isosurface or outisde, there are \(2^{8} = 256\) possible combinations of vertices. Luckily (again!) only \(14\) of those are actually relevant, since all the other are just rotations or mirrorings of these “base” cases.

Figure 1: The fundamental cases of the marching cubes algorithm

Cave Generation

Starting slow

Before going 3D, I thought starting with a 2D cave generator would have been a good idea to first learn the basics of Unity. That’s when I encountered this great video series, which helped me learn a lot.

First steps into Unity

The first thing I got to work was generating a 2D map, made of squares which are randomlz chosen to be black or white (Figure 2).

Figure 2: A first approach to a random generated map

As you maybe already noticed, this set-up really looks like the starting point of a Conway’s Game of Life game. Indeed, the procedural map generation will be based on cellular automation, but not on the set of rules Conway had defined.
The map generation is based on a seed system, and I also added some variables to play with for the map generation, such as a fillPercent field, with whom I could easily manage how much i wanted the map to be filled.

The rule I’ve used are failry simple:

If the cell as more then 4 alive neighbour cells, make it alive;
If it has less than 4 alive neighbour cells, make it die;
If it has exactly 4 alive neighbours, let it be as it is;

This precise rule is what I found working the best to generate a cave-resembling shape. Moreover, I’ve added some tweaks to the code to make sure that near walls more cells become alive, and ensure that the more we go towards the end of the cave, the thicker the walls are. I decided to apply the rule for 5 iterations, which seemed to work just fine. Some results can be obvserved below.

Figure 3: Some of the results of the algorithm: on the left, two results displayed on a 128x64 grid; on the right, two bigger grids (500x250 on the top, 1000x500 on the bottom). All measures in "squares".

In the end I played around with grid dimension, seeds, number of iterations and also tried changing the rule threshold (which was previously 4), but I didn’t get any result worth mentioning.