Go Channels — Behind the Scenes

Let me start by saying that Go Channels are GREAT! They are used for communication, allowing sending data between goroutines, and for synchronization, so that one goroutine doesn’t get ahead of another. But, how can a single data structure do all of this? Let’s go behind the scenes of Go Channels

Go Channels

A channel is a data type that provides us for free the following unusual and really interesting characteristics:

• Goroutine-safe
• Communicates values between goroutines in a FIFO manner
• Blocks and unblocks goroutines
• Provides an optional buffer

Channels in action

Using channels we convert a sequential and potential slow application into a fast, concurrent, and robust application. Let’s see a classic example implementing a program that will read tasks from a generator and will process them concurrently using workers

Channels example — main function

The main function creates a certain number of workers (goroutines) that will process tasks and it will wait until the done channel gets a notification. The script is executed as: go run basics/channel.go 3 where 3 is the number of workers.

Channels example — generator function

The generator function creates and returns a buffered channel and generates 5 tasks every x millisecond. Each task is sent to the channel and when it finishes generating them it closes the channel.

Channels example — worker function

The worker function is listening to the Task channel, blocking the goroutine until the channel has a task to process. Once the worker is done, it notifies that through the done channel.

Here is the result of running the script with one and three workers:

Channels example — output

Channels explained

As you saw in the example, we have communication and concurrency with no locks, no condition variables, and no callbacks. We should realize that by using more workers we have less blocking code and depending on the number of available cores we can have parallelism. Go relies on CSP (Communicating Sequential Processes) concurrency model. By default, a channel buffer size is zero and it is called unbuffered channel, and whatever a goroutine writes to the channel is immediately available to be read by another goroutine.

When the buffer size is non-zero, the producer goroutine is not blocked until after the buffer is full, even when there are no worker goroutines ready to process a task. Only when the buffer is full, the next producer goroutine will be blocked.

Let’s see how this works behind the scenes (step by step). From the Go channel code at chan.go we get these three main structs:

Channels main structs

hchan, waitq, and sudog structs do the magic behind the Go channels.

Creating a channel

We can create buffered channels which have a non-zero capacity or, unbuffered channels which allows synchronous communication. Let’s focus on buffered channels that can be created as c := make(chan Task, 3). The make built-in function will generate the hchan struct as follow:

Diagram 1: hchan, waitq, and sudog

In the Diagram 1 we can see that there is a buf buffer which points to circular buffer using an unsafe.Pointer and there are a sendx and recvx indexes. The circular buffer stores elements of the channel’s type. Of course, it has enqueue and dequeue operations. Also, we can see that there are sendq and recvq queues that contain a list of senders and receivers goroutines. Finally, there is a mutex lock that give us the goroutine-safe feature.

The hchan struct is allocated on the heap and the built-in make function returns a pointer to the struct. That’s why we pass the channels between functions as value semantics

Sending and receiving values

Do not communicate by sharing memory. Share memory by communicating.

Communication between two Goroutines

Let's see how two goroutines communicate with each other using a channel. g1 is producing a task and sending it to the channel, and g2 is receiving the task. What I will describe below applies to multiple senders and receivers. Remember that we’re using a buffered channel with a capacity of 3 elements (of type Task). Here the steps that g1 performs to send a task:

1. acquires the lock to modify the hchan struct
2. enqueue a copy of the task in the circular buffer
3. releases the lock

Notice that the actual enqueue is a memory copy of the task into the buffer slot. Then g2 wants to receive from the channel, so it repeats the g1 steps: acquires the lock, dequeue, and releases the lock. Again, the dequeue is a memory copy operation. These copy operations allow us to share memory by communicating.

Let’s say that processing a task takes much more time than producing it, so g2 is busy while g1 sends 3 tasks. In that case, the buffer is full, theng1 sends the fourth task to the channel… here is where the magic happens.

Pause and resume goroutines

g1 is paused until there is a slot in the buffer, in other words, when g2 is free to receive a new task. Of course, the task that g2 will receive comes from the buffer since the communication is in a FIFO manner. How does g1 gets paused and then resumed? the answer is Go's runtime scheduler Let’s “pause” here :) and do a brief description of how it works.

Runtime scheduler

The main goal of the runtime scheduler is to handle user-space threads (goroutines) which are lightweight compared with OS threads. They are less expensive consuming resources. The runtime scheduler manages goroutines by multiplexing N goroutines to M kernel thread.

Diagram 2: Runtime scheduler

As shown in the image above, there are 3 types of goroutines: running which are the ones that are actually being running in an OS thread; runnable which are the ones that are ready to be executed but they are paused; and blocked which are the goroutines that are not ready for execution. The runtime scheduler multiplexes the goroutines to a few OS threads.

The MxN scheduling is defined with 3 structures: M representing an OS thread; G the goroutine; and P a context for scheduling. Ps holds the Gs that are runnable in a run queue. So, in order for M to executes a G, the OS thread should have the context P.

Diagram 3: MxN scheduling

Pausing a goroutine

Ok, after the explanation about the runtime scheduler, let’s continue talking about channels and goroutines. So the buffer is full, g2 is busy and g1 sends a new task, but since there is no room for that task, g1 should be paused. Here, a gopark call to the runtime scheduler is performed so that g1 status changes from running to waiting and the association between g1 and M is removed. Notice that the OS thread is freed to run another goroutine from the P run queue.

As we saw in the Diagram 1 there are 2 waiting queues at hchan. These queues are pointing to the waitq struct that points to a sudog struct. This sudog struct stores the goroutine itself (in this case g1) and its element (in this case a Task).

Resuming a blocked goroutine

There comes the time when g2 finishes processing a task, and it is ready to pick up another from the buffer. In this scenario, g1 can be resumed. As we said, the goroutine g1is at the sendq queue in a sudog struct. Here is what g2 does:

1. Dequeues an element from the buffer
2. Pops the top sudog from the sendq queue
3. Enqueues the element (a Task) into the buffer
4. Calls goready function in the runtime scheduler to set g1 as runnable

Since g1 is set as runnable, it will be executed eventually.

Receiver coming first on an empty channel

What happens if g2 come first? well, pretty much the same, in this case g2 goes to recvq queue and waits. Then g1 generates a Task and… well, we can imagine that g1 enqueue the Task into the buffer and calls goready(g2) in the scheduler. Right? Not exactly, Go is smarter and performs optimizations whenever it is possible.

Since the sudoq at recvq queue has a pointer to the element that q2 is waiting to process, g1 writes the Task directly to that memory location. With this approach g2 doesn’t need to lock the buffer and we have one fewer memory copy.

Conclusion

I started this article by saying that Go Channels are great. I hope you think the same now that we have a better understanding of what they are behind the scenes. To finalize the article let’s see how this data type has the features that we listed:

• Goroutine-safe: with the hchan's mutex
• Communicates values between goroutines in a FIFO manner: with the circular buffer
• Blocks and unblocks goroutines: with the help of the runtime scheduler and hchan's sendq and recvq sudog queues
• Provides an optional buffer: with the hchan's buffer. Keep in mind that we have buffered and unbuffered channels.