Compiler Speed Trials

[boriel]

First of all, a big Thank you. Wow! What an impressive work! :o

I know that ZX Basic is amazing, but I was wondering how it stood up to other basic compilers that were around for use on the ZX Spectrum. We know that Hisoft basic was pretty fast, for example, and LCD mentioned another compiler the other day that was pretty amazing too.

Let me borrow from an article in Crash Magazine: http://www.crashonline.org.uk/19/compilers.htm

In this article, Simon Goodwin talks about several compilers. Hisoft Basic isn't one of them - it wasn't out yet. He doesn't list the benchmarks, either; but they can be interpolated from this:

Code:

Benchmark BM1 : A null-action FOR, REPEAT or DO loop, executed
                1000 times.

Benchmark BM2 : A  null-action explicitly-coded loop  executed
                1000 times.

Benchmark BM3 : BM2 plus A=K/K*K+K-K in the loop.

Benchmark BM4 : BM2 plus A=K/2*3+4-5 in the loop.

Benchmark BM5 : BM4  plus  a branch to null-action  subroutine
                from inside the loop.

Benchmark BM6 : BM5  plus  an array declaration  M(5),  and  a
                null-action  FOR  loop (of 1-5)  also  in  the
                loop.

Benchmark BM7 : BM6 plus M(L)=A in this 1-5 loop.

Benchmark BM8 : A  square  function,   log  function  and  sin
                function  in  an  explicitly-coded  FOR  loop,
                repeated 100 times.

Benchmark BM9 : Prime  numbers in the range 1-1000 are printed
                to the screen,  calculated in an outer loop of
                1000 and an inner loop of 500,  with no tricks
                at  all.  This  is  a very  bad  prime  number
                routine  indeed,  but a very useful basis  for
                inter-machine,    interpreter   and   compiler
                comparisons.

Simon didn't use Benchmark 9, and I can see why - it's not clearly specified. BM1 to BM8 are pretty clear, however.

The above benchmarks are interesting. I'm somewhat surprised of BM7 I didn't suppose array handling was so slow.
BTW do these compilers handle multiple-dimentions arrays?

First up, passing all the benchmarks and more, clearly Boriel's work is by far the most flexible and comprehensive compiler available. It blows the spots off everything else in terms of WHAT it can compile, and all credit to him for creating it. It is excellent!

:oops: Thank you. I now fell more motivated!!! :twisted:

In terms of performance, it's pretty amazing, too. It's the second fastest of all the compilers listed here. Only ZIP goes faster, generally. BM7 is a little disappointing, in that the produced code seems to be slower than both MCODER 2 and Zip by a quite significant margin. Perhaps some examination of array handling code could improve this. With version 1.25 beta, sadly, I couldn't use -O3 as an option - the programs all failed to compiler with this option enabled, so I couldn't see if peephole optimization would make a difference.

Yes, -O3 definitely makes a difference in array-access speed :!: This is something I'm currently fixing. It seems I reintroduced 2 old bugs back (one in the peephole and another on comparators already fixed). I'm currently working on them.

It's worth noting that most On Spectrum compilers refused to deal with floating point numbers. In this roundup, only Softek FP could do it, and that barely faster than Basic. Boriel's compiler blew me away with the FP result, frankly. I had to check to see if it was doing it correctly, it was so amazing! There might be some sneaky optimization happening, but printing the numbers as it created them did seem to work fine.

In short: Solid and well optimized. Seems to be slow in BM7 (array handling). Amazing BM8 (Floating Point) speed!

I will edit this post as and when I get around to testing a couple of other compilers. Hisoft Basic and Tobos - both able to do floating point code - are certainly going to be looked at!

I'm happy to read this.
For the FP, it's somewhat odd: I just use constant folding (precalculation) and ROM-CALC for that. Please, check the FP results are right... I mean do you print the FP calculation result on the screen? Do they match? How strange... :| I like this FP result, but... as you, I'm too surprised.