Understanding C++ Move and Perfect Forwarding

* This blog post is an English translation of an article originally published in Japanese on March 13, 2016.

This term, our internal study groups—which are winding down with less than a month left—include a Mathematical Optimization study group and an Effective Modern C++ reading group. This article is based on materials from the latter, specifically when we delved into Chapter 5 of Effective Modern C++.

C++11 was a game-changer, introducing a wealth of features that made the language significantly more convenient compared to its predecessor, C++03. C++14 further refined things, adding useful elements that either didn’t make it into C++11 or were initially overlooked. Our Effective Modern C++ reading group uses the book as a guide to explore these pivotal C++11/14 enhancements.

It’s already 2016, and modern compilers like GCC, Clang, and MSVC++ generally support C++14. It’s safe to say that a compiler unable to utilize C++14 hardly deserves to be called a C++ compiler (though this is my personal take, not a company-wide consensus). If C++14 is a modern machine, C++11 feels like a post-war tool, and C++03 an Edo-period relic (again, purely my personal feeling!).

Given this, an Effective Modern C++ reading group might seem a bit behind the curve. However, these topics are often superficially understood, so I gained a tremendous amount from this study group. In particular, std::move and std::forward, which we’re discussing this time, are—in my experience—the most frequently cited useful C++11 features that many don’t fully grasp. While there are undoubtedly many excellent articles on these topics, I believe different perspectives can be valuable. So, I decided to write this article to solidify what I’ve learned.

On the other hand, C++ is famous for a culture where critiques can be quite sharp—at times feeling like (figuratively) “hatchets being thrown.” While I believe this article is largely sound, I’m not fully confident it’s immune to such scrutiny… (as mentioned, these are difficult topics). If you find any errors, please comment on the article or contact me directly on Twitter: @LWisteria.

Terminology Review

Before diving into move and forwarding, let’s have a quick review of some terms.

Rvalues and Lvalues

Expressions can be rvalues or lvalues. The strict definition is complex and difficult, but you can generally remember it as: “an expression whose address can be taken = lvalue, an expression whose address cannot be taken = rvalue.”

int z = f(x, g(y));

Here:

• x, y, z: lvalues
• g(y), f(x, g(y)): rvalues

Alternatively, though also not strictly precise, you can think of lvalues as named variables and rvalues as temporary objects.

References to lvalues and rvalues are called lvalue references and rvalue references, respectively. Usually, the former is written as T& and the latter as T&&.

int& y = x;    // lvalue reference
int&& y = g(x); // rvalue reference

I specifically said usually because even if declared with T&&, it can sometimes be an lvalue.

void f(int&& v)
{
    // v is declared as T&& (rvalue reference), but its address can be taken,
    // so v is actually an lvalue!
    // ∵ All function parameters are lvalues.
    std::cout << &v << std::endl;
}

The detailed mechanism for this will be explained later, but please keep in mind that when T&& appears in explanations from now on, it might not always be an rvalue (conversely, if you consider whether T&& is an rvalue or lvalue each time it appears, your understanding might improve).

Move and Copy

Move is a concept paired with copy. While copy means duplication, move means replacement (duplicate & destroy).
It’s like in RPG Maker, when you want to move an object, there’s no “move” command, so you “duplicate it, then delete the original.” That’s a move.

Constructors and assignment operators that implement this move are called move constructors and move assignment operators.

{
    const int y = x;
    const int z = x;
    int ret = 0;
    ret = y+z;
}

In the above, among the assignment operations =, the = in y=x must be a copy assignment. On the other hand, for z=x, ret=0, and ret=y+z, it should be fine for them to be move assignments (though copy assignment would also work) because we can discard the results of x or y+z (they won’t be used later).

Thus, the same “assignment” operation can have the meaning of copy or move. “To give it the meaning of move” is called (interpreting with | adopting) move semantics. (Actually, I haven’t explained this very strictly, so please refer to yoh-san’s “Move Semantics Aren’t Scary” (C++ Advent Calendar 2012, Day 15)(Japanese only) or similar resources.)

(Perfect) Forwarding

Forwarding means passing received arguments as-is to another function.

void f(int x)
{
    g(x);
}

And perfect forwarding means passing not only the value of the formal parameter but also other information (like its type) completely to the actual argument of another function. In the example above, if calling f(z); passes the information of z directly to g(), making it equivalent to calling g(z);, that is called perfect forwarding.

Therefore, this example does not achieve perfect forwarding. If z were declared as int& z;, then f(z) would result in g((int)z), with the reference & being silently removed.

Move

What is std::move?

std::move is the feature in C++ for realizing moves. However, contrary to its name, std::move does not perform a move.
std::move(x) only makes x movable.

What does “movable” mean? In the example given earlier:

{
    const int y = x; // not movable
    const int z = x; // movable
    int ret = 0;     // movable
    ret = y+z;       // movable
}

If we focus on the right-hand side here, the lvalue x might not be movable, but the rvalues 0 and y+z are always movable.
This makes sense because something is movable when its right-hand side can be discarded. Therefore, rvalues (≒ temporary objects) are always movable.

This means std::move can be realized by casting to an rvalue. Here’s how it looks in practice:

f(x+1);          // x+1 is an rvalue
f(x);            // x is an lvalue
f(std::move(x)); // move makes x an rvalue

Here, when I say std::move doesn’t actually move but only makes things movable (casts), it means that even using std::move might not result in a move.

When does this happen? When you std::move a const value.

static void Foo(const Pointer p)
{
    Pointer q(std::move(p)); // Moved, yet the copy constructor is called!
}

Why does this happen? Because almost all move constructors and assignment operators take non-const references as arguments.
To be more specific, a move is a process that “destroys the source of assignment.” Therefore, to manipulate (destroy) the argument (source of assignment), a non-const reference is required.

// Copy constructor
Pointer(const Pointer& f)
{
    this->ptr = f.ptr;
}

// Move constructor
Pointer(Pointer&& f)
{
    this->ptr = f.ptr;

    // Destroy after moving
    f.ptr = nullptr;
}

So, people like me whose hands automatically add const especially need to teach their hands that const values cannot be moved. Be careful.

Note, I said “almost” because, according to the standard, it is possible to take a const rvalue reference as an argument, but that’s just a copy, so it’s meaningless.
(Review: It’s fine to pass an rvalue to a function taking an lvalue reference. Conversely, you cannot pass an lvalue to a function taking an rvalue reference. Therefore, in most cases, a function taking a const rvalue reference is just inconvenient, and a function taking a const lvalue reference suffices.)

The True Nature of std::move

As mentioned earlier, std::move is just a function that casts to an rvalue.
So how does it cast to an rvalue? It looks like this:

template<typename T>
decltype(auto) move(T&& param)
{
    return static_cast<std::remove_reference_t<T>&&>(param);
}

What it’s doing is:

1. Since a function’s return value becomes an rvalue, we want a function that just returns the same value.
2. If the return value isn’t a reference, a copy (or move) will occur, so use decltype(auto) instead of auto.
3. Similarly, we want the argument to be a reference, but T& can only accept lvalues, so use T&& (T&& is not necessarily an rvalue reference, as will be explained later).
4. Ultimately, we want to return an rvalue reference. However, if T is an lvalue reference, then decltype(param) (which is T&&) would also become an lvalue reference. So, we explicitly cast to an rvalue reference (&&).

This is the flow.

So, std::move got its name from the meaning “to make movable,” but since what it does is “cast to an rvalue,” names like std::rvalue_cast were also proposed.
Personally, I think something like std::movable would have been good.

Sometimes Moves Aren’t Possible

Thus, since C++11, moves have become available, and it’s said that using moves makes things faster! However, it’s also said that one shouldn’t expect too much because that’s not always the case. This is because sometimes moves are not possible.

First of all, move operations (constructors, assignments) are only implicitly generated if they are not explicitly implemented AND all of the following conditions are met:

• Copy operations are not explicitly implemented.
• The destructor is not explicitly implemented.
• All fields are movable.

This means that for complex and large objects that you’d want to move, they often fail one of these conditions, so move operations usually don’t exist unless explicitly implemented.

Also, even if move operations exist, whether a move is lighter than a copy is implementation-dependent. For example, std::vector moves are indeed fast because they just involve pointer swapping, but std::array has to copy its contents, so move and copy are about the same.

It’s unlikely to become slower, but since it’s implementation-dependent, it’s possible for such move operations to exist.
So, it’s better not to assume that using a move will always be faster than a copy.

Furthermore, even if a move operation exists and is definitely lighter than a copy, algorithms requiring a strong exception safety guarantee (the property that if an exception occurs midway, the operation can always be rolled back to its state before the operation) cannot use the move operation unless it is noexcept (does not throw exceptions).

Enumerating cases where move operations are not effective is endless, so there’s no point in expecting too much.

That said, there are apparently cases where just switching to C++11 on a certain compiler resulted in a 1.x times speedup, so moves do often make things faster.
It feels like “Which is it?!”, but perhaps it’s best to have the mindset of, “I thought I might have done well on the final exam, but I’ll be hurt if the score is low, so I’ll just assume the score is low.”

Perfect Forwarding

What is std::forward?

std::forward is a function that “makes a call perfectly forwarded.”
As mentioned at the beginning, perfect forwarding means passing the complete type information of the original actual argument of a formal parameter to the actual argument of another function.

Consider a template function F that calls an overloaded function f and also groups other common processing.

In C++03 or earlier, you might write something like this:

template<typename T>
static void F(T v)
{
    // Do some common processing here
    f(v); // Also call the overloaded function
}

But this doesn’t achieve perfect forwarding.

Why? Because when F(x) is called, x is copied to v, so the argument passed to f refers to a different entity (though its content is copied, so it looks the same).
Besides, for objects with non-negligible copy costs, you want to avoid copies.

So, if copying is the problem, should we use references?

template<typename T>
static void F(T& v)
{
    f(v);
}

No, this fails from the start because you can’t pass rvalues.

Now, from C++11, rvalue references can be used!

template<typename T>
static void F(T&& v)
{
    f(v);
}

This allows rvalues to be passed. However, it’s still incomplete.

Why? Because, as mentioned in the review section, formal parameters are always lvalues. So, going through F prevents you from calling an f that takes an rvalue reference. You might think, “Okay, I’ll just use std::move,” but then you can’t call an f that takes an lvalue reference.

Therefore, to achieve perfect forwarding, you need to appropriately cast the actual argument of F: if it’s an rvalue, the argument to f should also be an rvalue; if it’s an lvalue, it should be an lvalue.
std::forward achieves this.

template<typename T>
static void F(T&& v)
{
    f(std::forward<T>(v));
}

Now, perfect forwarding is achieved. Happily ever after.

In short, std::forward<T>(v) is a function that casts v such that:

• If the original source of v was an rvalue, the argument becomes an rvalue.
• If the original source of v was an lvalue, the argument becomes an lvalue.

Just like std::move, std::forward is also just a cast; std::forward itself does not perform the forwarding.

Mechanism of std::forward and Reference Collapsing

Perfect forwarding using std::forward:

template<typename T>
static void F(T&& v)
{
    f(std::forward<T>(v));
}

I wrote that std::forward takes T and determines whether the original source was an rvalue or an lvalue. This means “the information about whether the original source was an rvalue or lvalue is contained in T.”

However, it’s not like a new type is added. Actually, when type deduction for T&& happens, the specification is such that if an lvalue is assigned, T becomes a reference, and if an rvalue is assigned, T becomes a non-reference.
For example:

template<typename T>
static void F(T&& v)
{
    f(std::forward<T>(v));
}

int x;
F(x); // x is an lvalue, so T is int&
F(1); // 1 is an rvalue, so T is int

It’s difficult to directly observe what T actually is, but you can find out using the TypeDisplayer technique.

So, std::forward<T> should behave as follows:

• When T is a reference, it determines that the original source of v was an lvalue and returns an lvalue reference.
• When T is a non-reference, it determines that the original source of v was an rvalue and returns an rvalue reference.

So how is such behavior achieved? This is where reference collapsing comes in.
You might wonder what reference collapsing is.

Simply put, it’s like saying, “Actually, at compile time, there are references to references, and these are converted to normal references! (What?! – AA omitted)” (AA refers to ASCII Art).
A reference to a reference is something that usually doesn’t compile, like:

int& && y = x;

You might think this wouldn’t come up, but for example, if you use templates and typedef/aliases like:

template<typename T>
struct L
{
    using Type = T&;
};

And you substitute a reference for T, it will appear at compile time. For example, if you substitute an rvalue reference (int&&) for T, Type becomes an lvalue reference to an rvalue reference (int&& &), right?

Reference collapsing is the process of converting such double references that occur at compile time into single references.

Since there are rvalue and lvalue references, there are 4 patterns, and they are converted according to the following rules:

In other words, for example, int&& && becomes int&&, and all others become int&.
If you actually try it, you’ll see this is indeed the case.
With this, std::forward can:

• Create an rvalue reference if a non-reference (int) or an rvalue reference (int&&) is substituted for T in T&&.
• Create an lvalue reference if an lvalue reference (int&) is substituted.

Therefore, std::forward can be implemented as:

template<typename T>
decltype(auto) forward(std::remove_reference_t<T>& param)
{
    return static_cast<T&&>(param);
}

How does this work? For int x;:

Case: forward(x) (lvalue passed)

• T becomes a reference (int&).
• remove_reference removes the reference, making it a non-reference (int), and the type of param returns to a reference (int&).
• T&& becomes int& && (rvalue reference to an lvalue reference), which collapses to an lvalue reference int&.
• So, overall, when an lvalue is received, its lvalue reference is returned.

Case: forward(1) or std::forward(std::move(x)) (rvalue passed)

1. T becomes a non-reference (int).
2. Since it’s already a non-reference, remove_reference does nothing, it remains a non-reference (int), and the type of param becomes a reference (int&).
3. T&& is evaluated as int&& and becomes an rvalue reference.
4. So, overall, when an rvalue is received, its rvalue reference is returned.

This achieves the desired behavior.

Note that there are actually only 4 situations where reference collapsing (i.e., references to references) occurs. Of these:

• typedef or aliases
• Type deduction in templates with T&&
  have already been explained.

The third one is that it actually occurs not just in templates but also in auto&& type deduction.

int x;
auto&& y = x; // An lvalue is assigned, so auto becomes int&, and y becomes int& &&, which collapses to int&.
auto&& z = 1; // An rvalue is assigned, so auto becomes int, and y remains int&&.

This is also difficult to observe directly, but you can confirm it with TypeDisplayer.

Thus, when using templates or auto for type deduction, && can become an lvalue reference instead of an rvalue due to reference collapsing.
This is as I wrote at the beginning: “&& is not necessarily an rvalue reference.” Such && involved in type deduction, which might actually become an lvalue reference, is sometimes called a universal reference.

And the last, fourth case, is with decltype.
decltype returns the type of the variable if given a variable name, and its lvalue reference if given an expression. So, for int x;:

• decltype(x) is int.
• decltype((x)) is int&.

This means:

template<typename T>
void f(T v)
{
    decltype((v)) a = v;
}

f<int&&>(1); // Here, T is int&&, so decltype((v)) is int&& &, and after reference collapsing, the type of a becomes int&.

This can happen.

Cases Where Perfect Forwarding Fails and Countermeasures

Now, we implemented perfect forwarding like this earlier:

template<typename T>
static void F(T&& v)
{
    f(std::forward<T>(v));
}

But this is actually a bit incomplete because it doesn’t support overloads with different numbers of arguments.

extern void f(int x);        // This can be called via F
extern void f(int x, int y); // This cannot be called via F

template<typename T>
static void F(T&& v)
{
    f(std::forward<T>(v));
}

To support cases with different numbers of arguments, let’s make it variadic.

template<typename... Ts>
static void fwd(Ts&&... params)
{
    f(std::forward<Ts>(params)...);
}

Now we can perfectly forward in any case!! Happily ever after!

Or so I’d like to say, but actually, even with this, there are 5 known patterns where perfect forwarding can fail, meaning the behavior differs when writing the same ??? in f(???) and fwd(???):

• Cases where a reference cannot be taken:
  • Bit-fields
  • Static (compile-time) constant fields without a definition
• Cases where type deduction fails or the result is unexpected:
  • NULL
  • Overloaded functions / template functions
  • Braced initializers

But don’t worry. There are countermeasures. Let’s look at them one by one.

First off. As we’ve seen so far, perfect forwarding requires taking a reference to the original argument (without a reference, you’d have to copy).
Rvalue references have made it possible to take references to almost anything, but there are actually two patterns where references cannot be taken.

Of the two patterns where references cannot be taken, the simpler one is bit-fields.

struct Foo
{
    std::uint32_t a: 28,
                  b: 4;
};

In most cases, you cannot take a reference to these.

Foo f;
std::uint32_t& a  = f.a; // Cannot take an lvalue reference
std::uint32_t&& b = f.b; // Cannot take an rvalue reference either

Why? Because references are hardware-wise similar to pointers and require an address, but you can’t get “the address 4 bits later,” for example. You can only get an address for the result of converting the bit-field to an integral type.

So, conversely, in cases where the address of the converted integral type suffices, meaning:

• Non-const reference
• The original struct is an rvalue

it’s considered okay to take a reference because the value isn’t being modified.

// const reference is OK
const std::uint32_t& a = f.a;
const std::uint32_t& b = f.b;

// If the original struct is an rvalue, non-const reference is also possible
std::uint32_t&& c = std::move(f).a;

Both become references pointing to the entity (address) of the converted integral type.

So, to be more precise, if the original struct is an lvalue, you cannot take a non-const reference to a bit-field, and in this case, perfect forwarding is also not possible.

Foo foo;
f  (foo.a); // OK
fwd(foo.a); // NG

To avoid this, you just need to explicitly create a copy.

Foo foo;
const uint32_t a = foo.a;
f  (a);
fwd(a);
foo.a = a;

The other pattern where a reference cannot be taken is a static (compile-time) constant field without a definition.

struct Foo
{
    static const std::size_t N = 10;
    static const std::string S;
};
// const std::size_t Foo::N = 10; // OK even without a definition
const std::string Foo::S = std::string("foo"); // This is a non-integral type, so a definition is also needed

For integral types, you can use Foo::N even without a definition. This is because due to const propagation, even without a definition (entity), the compiler must replace Foo::N with 10.
So, if you try to take a reference in this state, there’s no entity = no memory allocated, so you can’t get an address, meaning you can’t take a reference (it compiles but results in a link error).

This happens quite often, and I frequently encounter link errors when trying to perfect forward a constexpr. You might think, “Why does a compile-time constant need a definition?!” so be careful (though, if you provide a definition as told, it will work).

However, some compilers apparently work even without a definition, and this is supposedly allowed by the standard (but at least it doesn’t work with gcc/clang).

Other than these cases where a reference cannot be taken, there are cases where perfect forwarding fails even if a reference can be taken.
As we’ve seen all along, perfect forwarding is achieved by skillfully using type deduction. Conversely, if type deduction fails or the inferred result is different, perfect forwarding cannot be done. Three such patterns are known.

The simplest is when using NULL.
The entity of NULL is (usually) 0, so it’s inferred as an integral type (usually int) rather than a pointer whenever possible.

So, if you try to perfect forward NULL instead of a pointer, you’ll get a compile error.

static void f(int* adr);
fwd(NULL); // NG
f  (NULL); // NG

The solution is to use nullptr.

fwd(nullptr);

Let’s stop using NULL so that modern kids don’t say, “You’re still using NULL? You’re like an old man!”

Now, the next pattern is overloaded functions and template functions.
Assigning a function to a function pointer type variable to get a function pointer to a function:

void (*p)(const int) = f;

f(1);
p(1);

This has been an unchanging phenomenon since ancient times. But did you also know that when taking an overloaded function as a function pointer, the one whose type matches the receiving side is assigned?

static void f(const int val);
static void f(const char str[]);

void (*p)(const int val)    = f; // This is the f above
void (*q)(const char str[]) = f; // This is the f below

It feels a bit weird because it’s deducing the right-hand side from the left-hand side, unlike usual type deduction, but thanks to this, even if functions are overloaded, as long as the type is clear, you can distinguish and get the function pointer.

This is also the same when used as a function argument:

static void g(const int val);
static void g(const char str[]);

static void f(void (*p)(const int val));

f(g); // Since f only accepts a function that takes int, this g becomes the upper g

However, trying to perfect forward this will fail.

f(g);   // OK
fwd(g); // NG

This is because fwd is a function that accepts any type, so it cannot deduce g from the assignment target as usual.

The workaround is to provide type information on the source side, such as by casting.

fwd(g); // NG
fwd(static_cast<void(*)(const int)>(g)); // OK

Note that this problem is the same for template functions.

template<typename T>
static void g(const T val);

f(g);
fwd(g); // NG
fwd(static_cast<void(*)(const int)>(g)); // OK
fwd(g<int>); // OK

The last one is braced initializers. These, like function pointers, determine their type based on the assignment target.

std::vector<int> v = {0, 1, 2};
auto&&           i = {4, 5, 6}; // initializer-list

So, if you try to perfect forward, a problem arises where it cannot deduce what type it should be, similar to function pointers.

f({0, 1, 2});   // initializer-list
fwd({4, 5, 6}); // NG

And the workaround for this is also the same as for function pointers: just explicitly specify the type.

auto i = {4, 5, 6}; // This way, it's an initializer-list
fwd(i);
fwd(std::vector<int>({4, 5, 6}));

Summary

So, I’ve given a rough explanation of C++ move and forwarding.

It might seem complicated until you actually use it, but understanding these concepts becomes very important, especially when trying to design fast and convenient libraries.
If you can explain these things comprehensively, I think it’s fair to say you’ve graduated from C++er Level 1.

Author

YOSHIFUJI Naoki