A look at WebAssembly, the webs powerful new technology part 2

Continuing on with our article from last week, we will explore using WebAssembly (wasm) in our code by writing it by hand. WebAssembly in two different formats. A human readable text version called WebAssembly Text, that looks a lot like Lisp (a language I am very familiar with). And a binary format ending in .wasm. You can freely convert between the two, and most major browsers have a way of converting .wasm back to .wat for debugging purposes. The Mozilla Developer network (MDN) is a fantastic resource for learning WebAssembly in depth. Let’s see what the MDN has to say about wasm.

It is not primarily intended to be written by hand, rather it is designed to be an effective compilation target for source languages like C, C++, Rust, etc.

DW is right

Well… Don’t be discouraged by the MDN…. While WebAssembly is intended to be a compilation target, the basics are very understandable. And if Chris Sawyer can create all of the original roller coaster tycoon in assembly, I think we’ll be able to handle a few functions.

Our journey begins 🛣️

Let's dive into a simple example to grasp the components of a WebAssembly Text file. A WebAssembly text (wat) file will need to be compiled into a .wasm file before it can be run in the browser. WABT, the WebAssembly Binary Toolkit will allow us to do that. Go ahead and download it now if you are following along. Once you’ve downloaded it, it’s time to create the.wat file. Let’s start with one called empty.wat. Put this code inside of it…

(module) 

Each WebAssembly file will have its code contained within a module. An empty module need not have any code inside of it, so this represents the simplest WebAssembly text file we can write. We can compile empty.wat using wat2wasm, a tool we get from WABT, with the -v (verbose) flag. This gives us two things, a hex dump of the WebAssembly binary file displayed in a human readable format, and our compiled .wasm. Looking at the verbose output of our compilation, we see the two mandatory headers that are required for all WebAssembly modules, WASM_BINARY_MAGIC, and WASM_BINARY_VERSION. While the hex dump is outside of the scope of this tutorial, it’s an interesting detail that I thought was worth mentioning.

PS /home/deca/programming/webassembly/helloWorld> wat2wasm empty.wat -v 
0000000: 0061 736d                                 ; WASM_BINARY_MAGIC
0000004: 0100 0000                                 ; WASM_BINARY_VERSION

Congratulations, you’ve created your first compiled wasm file. But an empty module does not a program make, so lets look at how to do something a little more complicated. Our next file will contain a function to square numbers. It will be named square.wat and it looks like this…

(module
  ;; create the func square that can be passed 1 variable $num
  (func (export "square") (param $num i32) (result i32)

    ;; add the value of $num to the stack
    local.get $num
    local.get $num

    ;; multiply the two $num values on the stack together
    i32.mul))

In our second example, func declares that the following code encapsulated between its parenthesis will be related to a function. (export “square”) indicates the name of the function that our JavaScript will call, in this case “square”. We export it so that it becomes accessible to the JavaScript code we will write soon. Our param block indicates the value we will pass to the function has the type i32 (a 32 bit integer) and the variable name will be $num. Variables are prefixed with the $ symbol à la Bash and Perl. (result) tells us what type our returned value will be, also an i32. As of the time of writing, WebAssembly can only return single values from its functions.

Next we have our function body. We have two instructions in the body called local.get, and one 32 bit integer instruction called i32.mul. To understand how this will be used to calculate the square we must talk about the stack.

The Stack

WebAssembly works by manipulating up to 4GB chunks of memory using instructions. You can view the spec for the supported instructions here, or allow me to save time by explaining it now. In the above example the instruction local.get $num puts the value of $num onto the stack. The value of $num will be the integer value that JavaScript passes to our wasm function. If you imagine the stack as a linear list of values, it would look something like this…

 [num 0 0 0 0 0 0] 

Where every time local.get is called with some value, that value is added to the list or “stack”. When local.get $num is called again it puts another num on the stack making it look like this now…

[num num 0 0 0 0 0 0]

We then call i32.mul, which takes two values off the stack, multiplies them together, then pushes the result back onto the stack. You can image the stack looks like this now

[num^2 0 0 0 0 0 0]

Since there are no more instructions after i32.mul, the function exits and the last result stored on the stack is returned (which in this case is the result of multiplying num by itself). There are many other instructions we can write in WebAssembly, and most should be easy to intuit like f32.add, f64.const, local.set, etc. But we will just stick to our example for now.

So now we have a working square function, but how do we get our module to run in the browser? First lets compile our square.wat using wat2wasm.

wat2wasm square.wat 

Now that we have our square.wasm file, I need some way to run it. This will require a little bit of HTML and JavaScript. We will put this in an index.html.

<!DOCTYPE  html>
<html>
  <head>
    <meta  charset="utf-8">
    <title>Using WebAssembly</title>
  </head>
  <body>
    <script>
      WebAssembly.instantiateStreaming(fetch('square.wasm'))
      .then(obj  => {
        console.log(obj.instance.exports.square(10));
      });
    </script>
  </body>
</html>

Between the <script> tags of our index.html, the JavaScript glue code to call our WebAssembly module resides. First we instantiate the WebAssembly module with this line…

WebAssembly.instantiateStreaming(fetch('square.wasm'))

Our fetch function is pretty self explanatory and gets the square.wasm binary. The instatiateStreaming portion takes the response from the fetch function and creates a new WebAssembly module.

.then(obj =>

Handles our promise. Once the WebAssembly module is instantiated, this function will be called. The instantiated WebAssembly module is passed as an object called obj.

obj.instance.exports.square(10)

We get the exported function in our obj, then call the square function, which we exported in our original wat file. It passes 10 as an argument to the square function and that will become that value of $num that is put on the stack by local.get. The console.log() then prints the result returned by our square function. To access my index.html file, I’m using the VS Code extension live server, which automatically serves it to port 5500, and updates every time I save my project. When I open my index.html in the browser, right click the web page and inspect the HTML, then switch to the console tab I see 100 in the log.

Which just so happens to be the square of 10 so we know our square function is working. Easy Peasy.

Okay but what about Hello World?

Now that we know the basics on how to compile our wat files and call them from JavaScript, we can work on getting the phrase “Hello World!” printed to our console. But there is a problem. WebAssembly does not have a string type. It only has these integer types, and vectors (with a reference type in proposal)

• i32: 32-bit integer

• i64: 64-bit integer

• f32: 32-bit float

• f64: 64-bit float

• v128: 128 bit vector of packed integer, floating-point data, or a single 128 bit type

Unfortunately this means that we will have to leverage the JavaScript API if we want to display “Hello World!” but I will walk you through it. WebAssembly has a way to import functions from JavaScript so that they can be called inside the module. Our new example will be put in a file called round.wat. I’ll only describe the new parts.

(module
  ;; importing our JavaScript round function
  (import "math" "round" (func $round (param f64) (result i32)))
  
  (func (export "roundWasm") (param $num f64) (result i32)
    local.get $num
    ;; calling our JavaScript round function
    call $round
  )
)

Our first new line starts with import. In the import declaration we state that the WebAssembly module should expect a function named round in an object/module named math. The function declaration is similar to our square function, but instead of receiving and producing an i32, it receives an f64 and returns a i32 instead. This is because we will be rounding numbers with decimals instead of whole number like before. We can execute the round function using a new WebAssembly instruction called, call. It pops the variable num we put on our stack using local.get and rounds it.

Now lets move onto our index.html file. In it we add…

// Our import object
const roundObject = {
    math: {
        round: Math.round
    }
};
WebAssembly.instantiateStreaming(fetch('round.wasm'), roundObject)
    .then(obj => {
        // calling our roundWasm function which calls the JavaScript
        // Math.round function
        console.log(obj.instance.exports.roundWasm(1.2));
    });

As we can see a few things have changed. Fetch can take another parameter. In this case it is taking our roundObject which is a import object that specifies the JavaScript’s Math.round function. We know we need to pass it this import object because that is how we specified it in the .wat file

  (import "math" "round" (func $round (param f64) (result i32)))

Without the import object, or if the import object is misconfigured, the WebAssembly module won’t be able to locate and execute the necessary imported functions, causing errors during execution. After saving and compiling my round.wat file, and saving the changes to index.html, my web page will be updated. If I look at the console in the browser, we see the number 100 (the value of our square function) has now been replaced with the number 1 (the value of 1.2 rounded down).

So now we have a way to pass a JavaScript function into a WebAssembly module so that it can be called. We almost have enough to write out “Hello World!” But we need two final additions. Let’s take a look at them now by starting with a new file called hello.wat.

(module
    (import  "console"  "log" (func  $log (param  i32  i32)))
    (import  "js"  "mem" (memory  1)) ;; Import 1 page of memory
    
    ;; Data section where we store our string at offset 0
    (data (i32.const 0) "Hello World!")

    (func (export  "helloWorld")
        // load the requisite data on the stack
        i32.const 0  ;; offset 0
        i32.const 12 ;; length of string
        
        // call log which will be passed our offset and string length
        call  $log))

Just like before we declare our imports. The first is for a log function, and the second is for a JavaScript memory function (we’ll get to this in a second). We then declare the data our WebAssembly module will hold. In this case we are initializing the data to “Hello World!” at index 0 in our modules memory, also called an offset. You can think of WebAssembly memory as a linear array separate from the stack that just holds data. In our example, each character will be placed in a memory array looking kind of like this

      ['H', 'e', 'l', 'l', 'o', ' ', 'W', 'o', 'r', 'l', 'd', '!']
index   0    1    2    3    4    ...                          11

Finally we create our helloWorld function.

    (func (export  "helloWorld")
        i32.const 0  ;; offset 0
        i32.const 12 ;; length of string  
        call  $log))

Our helloWorld function will pass the numbers we need to the JavaScript log function, so that it can index into the array of memory to find our string. We declared our Hello World! string as starting at i32.const 0, so we will have to put that number on the stack. Hello World! is made of 12 characters including the white space and the !, so we put 12 on the stack as well. Then we call our $log function which will consume both numbers on the stack. Back in our index.html…

var memory = new WebAssembly.Memory({
    initial: 1
});

// our custom log function that will use
// the offset and length of the string to 
// index into the memory in the correct placed
function customLog(offset, length) {
    var bytes = memory.buffer.slice(offset,length);
    var string = new TextDecoder('utf8').decode(bytes);
    console.log(string);
};

// our import object with out custom log function
var consoleObject = {
    console: {
        log: customLog
    },
    js: {
        mem: memory
    }
};

WebAssembly.instantiateStreaming(fetch('hello.wasm'), consoleObject)
    .then(obj => {
        obj.instance.exports.helloWorld();
    });

We also have some new code. We create a new WebAssembly Memory object in JavaScript using…

var memory = new WebAssembly.Memory({
    initial: 1
});

This instantiates a new page of WebAssembly memory that is of size 1. Each page of memory is defined as 64KiB so that is how large our array of memory is as well. Next we create our custom logging function. We need to do this because console.log in JavaScript does not actually take an offset and length, but we defined a log function in our wasm module that does.

function customLog(offset, length) {
    var bytes = memory.buffer.slice(offset,length);
    var string = new TextDecoder('utf8').decode(bytes);
    console.log(string);
};

This log function takes the parameters we need. It takes the JavaScript memory buffer which the wasm module will use to store the string, and creates a new slice that will just be the characters for our hello world.

var string = new TextDecoder('utf8').decode(bytes);:

The TextDecoder decodes the bytes in our memory, which will take the number representation for each character in our string ‘72’, ‘69’, ‘76’, etc that our WebAssembly module created, and decode them to a UTF-8 string. Finally

console.log(string);

Prints the decoded string to the JavaScript console. Our consoleObject, creates the import hierarchy of “console”, and “log” for our customLog function, and we bind our customLog to it. We also add another line in the import object to use the JavaScript memory function (because remember we had two imports in our hello.wat file).

var consoleObject = {
    console: {
        log: customLog
    },
    js: {
        mem: memory
    }
}

Just like before we pass our import object to our fetch function

WebAssembly.instantiateStreaming(fetch('hello.wasm'), consoleObject)
    .then(obj => {
        obj.instance.exports.helloWorld();
    });

And when we look at our log in the browser…

Voilà! Hello World! is printed. So to recap, we put the characters in the string “Hello World!” in a piece of memory

      ['H', 'e', 'l', 'l', 'o', ' ', 'W', 'o', 'r', 'l', 'd', '!']
index   0    1    2    3    4    ...                          11

That memory was created for our wasm module by JavaScript using

var memory = new WebAssembly.Memory({
    initial: 1
});

The values in that piece of memory were the number representation for each character in “Hello World!”. The wasm module passed the offset and the length of the string to our JavaScript function called customLog. The customLog function selected the first 12 values characters (the length of our string) in memory which corresponded to the values for “Hello World!”. Our TextDecoder('utf8') decoded these values as a utf-8 string. And we printed that string to the console and saw the string “Hello World!”.

Bonus Content

Now for some bonus content. If we want, we can modify our WebAssembly and JavaScript to interpret the null terminator as the end of the string. That way we don’t have to pass the explicit length of our string to our customLog function. Here are the changes to the hello.wat file, saved in a file called hello2.wat that reflect this…

(module
    (import  "console"  "log" (func  $log (param i32)))
    (import  "js"  "mem" (memory  1)) ;; Import 1 page of memory
    
    ;; Data section
    (data (i32.const 0) "Hello World!\00")
    
    (func (export  "helloWorld")
        i32.const 0  ;; offset 0
        call  $log))

As you can see, the log function now expects a single i32 parameter instead of two. And the Hello World string now contains the value “\00” which corresponds to the null terminator. Now let’s modify the customLog function index.html to reflect our wasm file updates.

var memory = new WebAssembly.Memory({
    initial: 1
});

function customLog(offset) {
    var bytes = new Uint8Array(memory.buffer);
    var end = offset;
    // iterate through the bytes in memory and stop
    // when it encounters the null terminator
    while (bytes[end] !== 0) {
        ++end;
    }
    var stringBytes = bytes.subarray(offset, end);
    var string = new TextDecoder('utf8').decode(stringBytes);
    console.log(string);
};

var consoleObject = {
    console: {
        log: customLog
    },
    js: {
        mem: memory
    }
};

WebAssembly.instantiateStreaming(fetch('hello2.wasm'), consoleObject)
    .then(obj => {
        obj.instance.exports.helloWorld();
    });

We take our memory that we created in JavaScript, and iterate over it. We count every byte until we hit the null terminator to get the length of our string. When JavaScript sees that it hits the null terminator it breaks the loop. The variable end becomes the length of our string. We then use that value of end as well as the offset we passed to get a slice of our memory from the offset (0) to the end of our string (12). When then decode these values as a utf-8 and print it to the console as before.

But while the null terminator is traditional for doing this, we can use whatever terminator we want. Here is a modified example of our log function looking for a tab instead of the null terminator

function customLog(offset) {
    var bytes = new Uint8Array(memory.buffer);
    var end = offset;
    while (bytes[end] !== 9) { // look for tab
        ++end;
    }
    var stringBytes = bytes.subarray(offset, end);
    var string = new TextDecoder('utf8').decode(stringBytes);
    console.log(string);
};

As well as the corresponding section of our hello2.wat file.

    ;; Data section
    (data (i32.const 0) "Hello World!\t") ;; Use a tab as the null terminator

If you are familiar with C, then you know that the null terminator is how C handles string lengths. This also means that if you don’t terminate the string correctly you run into all the issues that C has with strings. Fun!

It’s okay, No one will actually use this in production code. Right?

Lastly, another thing that we can do is just skip initializing the memory using JavaScript and create it in our WebAssembly module itself. Pay close attention to the fact we have to export this like a function so that it is accessible in JavaScript when we do it this way.

(module
    (memory $0 1)
    (export "memory" (memory $0)) ;; Exporting the memory

    (data (i32.const 0) "Hello World!\00")

    (func (export "helloWorld") (result i32)
        i32.const 0)
)

In our index.html you can access the wasm memory by doing

var memory = obj.instance.exports.memory

Once you have that, the logic for counting the bytes to the null terminator, decoding, and printing the string is exactly the same.

WebAssembly.instantiateStreaming(fetch('hello3.wasm'))
    .then(obj => {
        // Get the memory buffer
        var memory = obj.instance.exports.memory;
        var bytes = new Uint8Array(memory.buffer);
        var end = 0;
        while (bytes[end] !== 0) {
            ++end;
        }
        var string_slice = new Uint8Array(memory.buffer, 0, end);

        var string = new TextDecoder('utf8').decode(string_slice);

        console.log(string);
    });

Finally you might be wondering where I’ve been getting all these “magic” numbers like 9 and 0 for the tab and null terminator. This has to do with character encoding, and I have a section about it in my article titled Breaking the Snake: How Python went from 2 to 3, if you are interested in learning more.

that covers the basics of using WebAssembly in our JavaScript code. I hope you learned something new, and I’m excited to see where this technology goes in the future

Call To Action 📣

If you made it this far thanks for reading! If you are new welcome! I like to talk about technology, niche programming languages, AI, and low-level coding. I’ve recently started a Twitter and would love for you to check it out. I also have a Mastodon if that is more your jam. If you liked the article, consider liking and subscribing. And if you haven’t why not check out another article of mine! A.M.D.G and thank you for your valuable time.