ppci-build.py¶

This command can be used to construct build files (see Build system).

Take for example the stm32f4 led project build file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

<project name="Examples" default="all">
    <import name="ppci.buildtasks" />

    <target name="all" depends="burn2">
    </target>

    <property name="src" value="src" />
    <property name="arch" value="arm"/>

    <target name="burn2">
        <assemble source="startup_stm32f4.asm" target="thumb" 
            output="startup.o" />
        <compile target="thumb"
            sources="burn2.c3;arch.c3;../io.c3"
            includes="stm32f4xx.c3"
            output="burn2.o" />
        <link output="burn2.elf" layout="stm32f4.mmap"
            target="thumb"
            objects="startup.o;burn2.o" />
        <objcopy objectfile="burn2.elf"
            imagename="flash"
            format="hex"
            output="burn2.hex" />
    </target>

</project>

To build this project, run ppci-build.py in the same directory:

$ cd test/data/stm32f4xx
$ ppci-build.py

Or specify the buildfile:

$ ppci-build.py -f test/data/stm32f4xx/build.xml

ppci-c3c.py¶

Instead of relying on a build system, the C3 compiler can also be activated stand alone.

$ ppci-c3c.py --target arm examples/snake/game.c3

ppci-asm.py¶

Invoke the assembler.

ppci-hexutil.py¶

Utility to handle hex files.

Introduction¶

As an example language, the c3 language was created. As pointed out clearly in c2lang, the C language is widely used, but has some strange contraptions:

• The include system. This results in lots of code duplication and file creation. Why would you need filenames in source code?
• The comma statement: x = a(), 2; assigns 2 to x, after calling function a.
• C is difficult to parse with a simple parser. The parser has to know what a symbol is when it is parsed.
• Etc...

For these reasons (and of course, for fun), C3 was created.

Example¶

The hello world example in c3:

module hello;
import io;

function void main()
{
    io.println("Hello world");
}

Language reference¶

module pkg1;
import pkg2;

module pkg2;
import pkg1;

module example;

function void compute()
{
}

function void main()
{
    main();
}

module example;

var int global_var;

function void compute()
{
    var int x = global_var + 13;
    global_var = 200 - x;
}

module example;
var int number;
var int* ptr_num;
type int* ptr_num_t;
var ptr_num_t number2;

module example;

function void compute(int a)
{
    var int b = 10;
    if (a > 100)
    {
        b += a;
    }

    if (b > 50)
    {
        b += 1000;
    }
    else
    {
        b = 2;
    }
}

module example;

function void compute(int a)
{
    var int b = 10;
    while (b > a)
    {
        b -= 1;
    }
}

module example;

function void compute(int a)
{
    var int b = 0;
    for (b = 100; b > a; b -= 1)
    {
        // Do something here!
    }
}

Projects¶

The root element of a build file is the project tag. This tag contains a name and optionally a default target attribute. When no target is given when building the project, the default target is selected.

Targets¶

Like make, targets can depend on eachother. Then one target is run, the build system makes sure to run depending targets first. Target elements contain a list of tasks to perform.

Tasks¶

The task elements are contained within target elements. Each task specifies a build action. For example the link task takes multiple object files and combines those into a merged object.

buildfunctions module¶

This module contains a set of handy functions to invoke compilation, linking and assembling.

ppci.buildfunctions.assemble(source, target)¶

Invoke the assembler on the given source, returns an object containing the output.

ppci.buildfunctions.bf2ir(source)¶

Compile brainfuck source into ir code

ppci.buildfunctions.bfcompile(source, target, lst_file=None)¶

Compile brainfuck source into binary format for the given target

ppci.buildfunctions.c3compile(sources, includes, target, lst_file=None)¶

Compile a set of sources into binary format for the given target

ppci.buildfunctions.c3toir(sources, includes, target)¶

Compile c3 sources to ir code using the includes and for the given target

ppci.buildfunctions.construct(buildfile, targets=())¶

Construct the given buildfile. Raise task error if something goes wrong

ppci.buildfunctions.fix_file(f)¶

Determine if argument is a file like object or make it so!

ppci.buildfunctions.fix_target(tg)¶

Try to return an instance of the Target class

ppci.buildfunctions.get_compiler_rt_lib(target)¶

Gets the runtime for the compiler. Returns an object with the compiler runtime for the given target

ppci.buildfunctions.ir_to_code(ir_modules, target, lst_file=None)¶

Translate the given list of IR-modules into object code for the given target

ppci.buildfunctions.ir_to_python(ircode, f)¶

Convert ir-code to python code

ppci.buildfunctions.link(objects, layout, target, lst_file=None, use_runtime=False)¶

Links the iterable of objects into one using the given layout

ppci.buildfunctions.objcopy(obj, image_name, fmt, output_filename)¶

Copy some parts of an object file to an output

ppci.buildfunctions.optimize(ircode, do_verify=False)¶

Run a bag of tricks against the ir-code. This is an in-place operation!

Yacc¶

// .. automodule:: yacc

Burg¶

reg -> ADDI32(reg, reg) 2 (. add NT0 NT1 .)
reg -> MULI32(reg, reg) 3 (. .)

reg -> ADDI32(MULI32(reg, reg), reg) 4 (. muladd $1, $2, $3 .)

[result] -> [tree] [cost] [template code]

C3 Front-end¶

For the front-end a recursive descent parser is created for the c3 language. This is a subset of the C language with some additional features.

class ppci.c3.Lexer(diag)¶

Generates a sequence of token from an input stream

class ppci.c3.Parser(diag)¶

Parses sourcecode into an abstract syntax tree (AST)

class ppci.c3.CodeGenerator(diag)¶

Generates intermediate (IR) code from a package. The entry function is ‘genModule’. The main task of this part is to rewrite complex control structures, such as while and for loops into simple conditional jump statements. Also complex conditional statements are simplified. Such as ‘and’ and ‘or’ statements are rewritten in conditional jumps. And structured datatypes are rewritten.

Type checking is done in one run with code generation.

class ppci.c3.Builder(diag, target)¶

Generates IR-code from c3 source. Reports errors to the diagnostics system.

Brainfuck frontend¶

The compiler has a front-end for the brainfuck language.

class ppci.bf.BrainFuckGenerator¶

Brainfuck is a language that is so simple, the entire front-end can be implemented in one pass.

IR-code¶

The intermediate representation (IR) of a program de-couples the front end from the backend of the compiler.

See IR-code for details about all the available instructions.

Optimalization¶

The IR-code generated by the front-end can be optimized in many ways. The compiler does not have the best way to optimize code, but instead has a bag of tricks it can use.

class ppci.transform.ModulePass¶

Base class of all optimizing passes. Subclass this class to implement your own optimization pass

class ppci.mem2reg.Mem2RegPromotor¶

Tries to find alloc instructions only used by load and store instructions and replace them with values and phi nodes

class ppci.transform.LoadAfterStorePass¶

Remove load after store to the same location.

[x] = a
b = [x]
c = b + 2

transforms into:

[x] = a
c = a + 2

class ppci.transform.DeleteUnusedInstructionsPass¶

Remove unused variables from a block

class ppci.transform.RemoveAddZeroPass¶

Replace additions with zero with the value itself. Replace multiplication by 1 with value itself.

class ppci.transform.CommonSubexpressionEliminationPass¶

Replace common sub expressions with the previously defined one.

Back-end¶

The back-end is more complicated. There are several steps to be taken here.

1. Canonicalization
2. Tree creation
3. Instruction selection
4. register allocation
5. Instruction emission
6. TODO: Peep hole optimization?

def IMUL64rr : RI<0xAF, MRMSrcReg, (outs GR64:$dst),
                                   (ins GR64:$src1, GR64:$src2),
                   "imul{q}\t{$src2, $dst|$dst, $src2}",
                   [(set GR64:$dst, EFLAGS,
                       (X86smul_flag GR64:$src1, GR64:$src2))],
                   IIC_IMUL64_RR>,
                TB;

<insn> BC
{
  <decode>
  {
    %ID: {0x7495, 0x0483}
    %cond_code: { %OPCODE1 & 0x7F }
    %dest_address: { %OPCODE2 }
  }
  <schedule>
  {
    BC1(PF, w:ebus_addr, w:pc) |
    BC2(PF, w:pc), BC3(IF) |
    BC4(ID) |
    <if> (condition[cond_code])
    {
      BC5(AC) |
      BC6(PF), BC7(ID), BC8(RE) |
      BC9(EX)
    }
    <else>
    {
      k:NOP(IF), BC10(AC, w:pc) |
      BC11(PF), BC12(ID), BC13(RE) |
      k:NOP(ID), BC14(EX) |
      k:NOP(ID), k:NOP(AC) |
      k:NOP(AC), k:NOP(RE) |
      k:NOP(RE), k:NOP(EX) |
      k:NOP(EX)
    }
  }
  <operate>
  {
    BC1.control: { ebus_addr = pc++; }
    BC2.control: { ir = mem[ebus_addr]; pc++ }
    BC10.control: { pc = (%OPCODE2) }
  }
}

patterns
  nullary is any of [ HALT NEG COM SHL SHR READ WRT NEWL NOOP TRA NOTR ],
    which is op = 0 & adr = { 0 to 10 }
constructors
  IMULb        Eaddr            is      (grp3.Eb;    Eaddr) & IMUL.AL.eAX

type word = card(16)
type absa = card(9)
type disp = int(4)
type off = int(6)
mem PC[1,word]
mem R[16,word]
mem M[65536,word]
var L1[1,word]
var L2[1,word]
var L3[1,word]
mode register(i:card(4)) = R[i]
  syntax = format(”R%s”, i)
  image = format(”%4b”, i)
mode memory = ind | post | abs
mode ind(r:register, d:disp) = M[r+d]
  update = {}
  syntax = format(”@%s(%d)”, r.syntax, d)
  image = format(”0%4b%4b0”, r.image, d)
mode post(r:register, d:disp) = M[r+d]
  update = { r = r + 1; }
  syntax = format(”@%s++(%d)”, r.syntax, d)
  image = format(”0%4b%4b1”, r.image, d)
mode abs(a : absa) = M[a]
  update = {}
  syntax = format(”%d”, a)
  image = format(”1%9b”, a)
op instruction( i : instr )
  syntax = i.syntax
  image = i.image
  action = {
    PC = PC + 1;
    i.action;
  }
op instr = move | alu | jump
op move(lore:card(1), r:register, m:memory)
  syntax = format(”MOVE%d %s %s”, lore, r.syntax, m.syntax)
  image = format(”0%1b%4b%10b”, lore, r.image, m.image)
  action = {
    if ( lore ) then r = m;
    else m = r;
    endif;
    m.update;
  }
op alu(s1:register, s2:register, d:reg, a:aluop)
  syntax = format(”%s %s %s %s”, a.syntax, s1.syntax, s2.syntax, d.syntax)
  image = format(”10%4b%4b%4b%2b”, s1.image, s2.image, d.image, a.image)
  action = {
    L1 = s1; L2 = s2; a.action; d = L3;
  }
op jump(s1:register, s2:register, o:off)
  syntax = format(”JUMP %s %s %d”, s1.syntax, s2.syntax, o)
  image = format(”11%4b%4b%6b”, s1.image, s2.image, o)
  action = {
   if ( s1 >= S2 ) then PC = PC + o;
   endif;
  }
op aluop = and | add | sub | shift;
op and() syntax = ”and” image = ”00” action = { L3 = L1 & L2; }
op add() syntax = ”add” image = ”10” action = { L3 = L1 + L2; }
op sub() syntax = ”sub” image = ”01” action = { L3 = L1 - L2; }

Communication¶

Join the #ppci irc channel on freenode!

Running the testsuite¶

To run the unit tests with the compiler, use pytest:

$ pytest -v test/

Or use the unittest module:

$ python -m unittest discover -s test

In order to test ppci versus different versions of python, tox is used. To run tox, simply run in the root directory:

$ tox

Building the docs¶

The docs can be build locally by using sphinx. Make sure that ppci is on your PYTHONPATH

$ export PYTHONPATH=your_ppci_root
$ cd docs
$ sphinx-build -b html . build