News

News

04/07/2020
Krystian's March Update

The Rundown Due to the COVID-19 pandemic, my classes have been moved online. It’s certainly an interesting way to teach, but admittedly I can’t say it’s enjoyable or effective. However, it has given me a lot of time to work on various projects, which is a reasonable trade-off (at least in my opinion). I got quite a bit done this month due to the substantial increase in leisure time, and was able to work on several projects that previously didn’t fit into my schedule. Boost.StaticString I spent the first few days of March putting the finishing touches on Boost.StaticString in preparation for the release of Boost 1.73.0, mostly consisting of housekeeping tasks, but also some bug fixes for certain compiler configurations. In particular, a problem arose with GCC 5 regarding its constexpr support, two of which impede using basic_static_string during constant evaluation: throw expressions, and non-static member functions whose return type is the class they are a member of. With respect to the former, consider the following: constexpr int throw_not_evaluated(bool flag) { if (flag) throw 1; return 0; } constexpr int const_eval = throw_not_evaluated(false); View this on Godbolt It is helpful to first establish what the standard has to say regarding the above example. Looking at [dcl.constexpr] p3, we see that throw_not_evaluated contains no constructs that are explicitly prohibited from appearing within a constexpr function in all contexts. Now taking a look at [expr.const] p2 we see: A conditional-expression e is a core constant expression unless the evaluation of e, following the rules of the abstract machine, would evaluate one of the following expressions: […] a throw-expression Boiling down the standardese, this effectively says that throw_not_evaluated(false) is a constant expression unless, when evaluated, it would evaluatethrow 1. This would not occur, meaning that throw_not_evaluated(false) is indeed a constant expression, and we can use it to initialize const_eval. Clang, later versions of GCC, and MSVC all agree on this and compile it without any complaints. However, GCC 5 with the -std=c++14 flag fails to do so, citing: error: expression ‘<throw-expression>’ is not a constant-expression Sounds excusable to me! GCC 5.1 was released in 2015 after all, so you can’t expect every feature to be implemented less than a year after C++14 was finalized, right? It all sounded sane to me, but before going ahead and disabling constexpr for functions that were potentially-throwing, I decided to try a small variation of the original: struct magic { constexpr magic(bool flag) { if (flag) throw 1; return; } constexpr operator int() { return 1; } }; constexpr int magic_eval = magic(false); View this on Godbolt What? It miraculously works. Further, if we construct a magic object within a constexpr function: constexpr int throw_not_evaluated(bool flag) { return magic(flag); } View this on Godbolt What? Gathering the remnants of my sanity, I lifted the BOOST_STATIC_STRING_THROW_IF macros out of all the potentially-throwing functions and replaced them with a class template (for GCC 5) or function template (for all other compilers) that can be constructed/called with the throw_exception<ExceptionType>(message) syntax. I also considered adding a throw_exception_if variation to hide the if statement within shorter functions, but on advisement from Vinnie Falco and Peter Dimov, I didn’t end up doing this to allow for better optimization. Moving on to a simpler GCC 5 issue (but significantly more annoying to diagnose), I discovered that substr was causes an ICE during constant evaluation. Took some time to get to the bottom of it, but I eventually figured out that this was because the return type of substr is basic_static_string. Unfortunately, the only remedy for this was to disable constexpr for the function when GCC 5 is used. Save for these two issues, the rest of the work that had to be done for StaticString was smooth sailing, mostly improvements to the coverage of the insert and replace overloads that take input iterators. In the future I plan to do an overhaul of the documentation, but as of now it’s ready for release, so I’m excited to finally get this project out into the wild. Boost.JSON Most of the time I spent on Boost.JSON in the past month was spent learning the interface and internals of the library, as I will be writing documentation on many of the components in the future. My primary focus was on the value_ref class, which is used as the value type of initializer lists used to represent JSON documents within source code. The reason that this is used instead of value is because std::initializer_list suffers from one fatal flaw: the underlying array that it refers to is const, which means you cannot move its elements. Copying a large JSON document is not trivial, so value_ref is used as a proxy referring to an underlying object that can be moved by an operation using an initializer list. This is achieved by storing a pointer to a function that will appropriately copy/move construct a value when called requested by the target. While looking at how value_ref works, I went ahead and added support for moving from a json::string, since up to that point all strings were stored as string_view internally, thus precluding the ability to move from a value_ref constructed from a json::string. There also was a bug caused by how value_ref handles construction from rvalues, in part due to the unintuitive nature of type deduction. Consider the following: struct value_ref { template<typename T> value_ref(const T&) { ... } template<typename T> value_ref(T&&) { ... } }; json::value jv; const json::value const_jv; value_ref rvalue = std::move(jv); // #1 value_ref const_rvalue = std::move(const_jv); // #2 In both #1 and #2, the constructor value_ref(T&&) is called. This certainly makes sense once you consider the deduction that is performed, however, by glancing at just the declarations themselves, it isn’t obvious, as we’ve all been taught that references to non-const types will not bind to a const object. Where this becomes a problem for value_ref is that the constructor taking a T&& parameter expects to be able to move from the parameter, so it internally stores a non-const void*. Converting a “pointer to const T” to void* isn’t permitted, so you get a hard error. The fix for this was fairly trivial. Standardization Most of my time this month was spent working on standardization related activities, which can be broken up into two somewhat separate categories: Fixing existing wording Working on papers Editorial issues I submitted a fair few pull requests to the draft repository, most of which were general wording cleanups and improvements to the consistency of the wording. With respect to wording consistency, I targeted instances of incorrect cv-qualification notation, definitions of terms within notes, cross-references that name the wrong subclause or are self-referential, and redundant normative wording. These kinds of issues are all over the standard, but I generally stay away from the library wording unless it’s absolutely necessary since it has a subtitle difference in the wording style compared to that of the core language. I ended up writing atool to make grepping for these issues a little easier, which is certainly an improvement over manually inspecting each TeX source. The wording cleanups are the most time consuming, but also are the ones I find the most enjoyable. They all follow the same principle of rephrasing an idea with more accurate wording while not changing the actual meaning – something that often proves to be a challenge. These generally start off as small issues I notice in the text, but then snowball into complete rewrites to make the whole thing consistent. Anyways, if it sparks your interest you can find the various editorial fixes I worked on here. P1997 Of my papers, P1997 is the one that I’m putting the most time into. In short, it proposes to make arrays more “regular”, allowing assignment, initialization from other arrays, array placeholder types, and many more features we bestow upon scalar and class types. The holy grail of changes (not proposed in the paper) would be to allow the passing of arrays by value without decaying to a pointer: fully cementing arrays as first-class types. Unlike the other proposed changes this isn’t a pure extension, so to make it remotely feasible existing declarations of parameters with array type (for the sake of brevity we will abbreviate them as PAT) would have to be deprecated in C++23, removed in C++26, and then reinstated in C++29. For this reason, we are undertaking this as an entirely different paper to allow for the pure extensions to be added in C++23, leaving the removal and reinstatement on the backburner. In order to bring convincing evidence that: The current semantics for PAT are unintuitive and merely syntactic sugar. The deprecation of PAT would not be significantly disruptive to existing codebases. Ted (the co-author) and I decided to see exactly how often PAT appear within normal C++ code. While codesearch.isocpp.org by Andrew Tomazos is a fantastic tool, the search we wanted to conduct was simply not possible with his tool, so we set out to create our own. We needed two things: A dataset Some tool to parse all the source files For the dataset, I wrote a tool to clone the top 4000 C++ Github repositories, and clean out all files that weren’t C++ source files (.cpp, .cxx, .cc, .h, .hpp, .hxx, .ipp, .tpp). Github has a nice API to search for repositories, but it unfortunately limits the results for a single search query to 1000 results, but since I was sorting them by start count, I was able to use it as an index to begin a new search query. After accidentally sending my Github credentials to Ted once and waiting 10 hours for all the repositories to be cloned, we had our dataset: 2.7 million source files, totaling around 32GB. To parse all these files, we opted to use the Clang frontend. Its AST matcher was perfectly suited to the task, so it was just a matter of forming the correct matchers for the three contexts a PAT can appear in (function parameter, non-type template parameter, and catch clause parameter). All the files were parsed in single file mode, since opening and expanding #include directives would make the processing of the files take exponentially longer. Forming the correct match pattern proved to be the most difficult part, as the syntax is not entirely intuitive and often times they simply didn’t find every PAT in our test cases. Doing a single file parse along with wanting to find every PAT possible and suppressing all diagnostics landed us in the land of gotchas. To start, the arrayType matcher only would match declarations that had an array declarator present, i.e. int a[] would be found but T a where T names an array type would not. Eventually I found decayedType, which did exactly what we wanted, so long as we filtered out every result that was a function pointer. This worked for function parameters and non-type template parameters, but not catch clause parameters. In the Clang AST, catch is categorized as a statement that encloses a variable declaration whose type is not considered to be decayed (as far as I could see) so we could only match parameters that used an array declarator. I don’t expect anyone to actually declare PATs in a catch clause, and after running the tool on the dataset exactly zero instances were found, so this is most likely a non-issue. Single file parsing introduced a number of issues all stemming from the fact that none of the #include directives were processed, meaning that there was a large number of types that were unresolved. Consider the following: #include <not_processed.h> using array = unresolved[2]; using array_ref = unk_array&; array_ref ref = {unresolved(), unresolved()}; void f(decltype(ref) param); For reasons that I don’t know, since unresolved has no visible declaration Clang reports that param has a decayed type. I suspect this is because diagnostics are suppressed and some recursive procedure used in the determination of the type named by array_ref returns upon encountering an error at the declaration of array and simply returns unresolved[2] as the type. If you know why this happens, don’t hesitate to ping me! I ended up tracking the number of PAT declarations that use an array declarator separately since I suspect that this number may end up being more accurate. Once the tool was ready to go and we started to run it on the dataset, we encountered issues of the worst kind: assertion failures. I suppose such errors could be expected when abusing a compiler to the extent that we did, but they weren’t particularly enjoyable to fix. I should mention that tool itself is meant to be run on a directory, so once an assertion failed, it would end the entire run on that directory. My initial solution to this was changing the problematic asserts to throw an exception, but the number of failures was ever-growing. Creating a new CompilerInstance for each file did somewhat remedy the situation, but didn’t fix it all. Eventually, we called it good enough and let it run over the entire dataset. Clang itself was in our dataset, with a nasty little surprise taking the form of infinitely recursive template instantiations and debug pragmas that would crash the compiler. Clang relies on the diagnostics to signal when the recursive instantiation limit is reached, but since those were disabled the thread would never terminate. Evil. Once the paper is finished, I’ll report the results in a future post. Implicit destruction This paper intends to substantially overhaul the wording that describes the interaction between objects and their storage. I originally brought this issue up several months back with this pull request in an attempt to fix it editorially, but it was deemed too large in scope. I finally got started on the paper and drafted up a direction to take, which will hopefully resolve the shortfalls of our current wording. This problem stems from the notion that objects don’t exist before their lifetime has started and after it has ended. This allows compilers to make a lot of assumptions and consequently leads to more optimized code, but the manner in which the wording was applied has severely crippled our ability to refer to “not-objects”. We want to be able to place restrictions on storage that an object used to occupy, but simply have no way for doing so. Thus, the direction I plan to take is to define the semantics of storage, allowing us to place restrictions on that storage even if no object exists within it. I don’t have too many of the core definitions completed yet as they require the most time to make them robust, but once that is hashed out, applying it where needed should be smooth sailing. Here is a short list of the main changes: Define what a region of storage, specify when it is acquired, released, and what kinds of objects may occupy it. Remove the storage duration property from objects, and effectively make that agnostic of the storage they occupy. Instead, associate storage duration with a variable. Dynamic storage duration can removed since such storage isn’t associated with a variable. Specify when reuse of storage occurs, and its effects upon the objects within that storage. Properly specify when pointers and expressions refer to storage while preserving the notion that they refer to objects (or functions). Specify that when control passes through the definition of a variable, storage is acquired, objects are created, and initialization is performed. Likewise, specify that exit from a scope, program, or thread causes objects within the storage to be destroyed (if any), and the storage to be released. It’s a big undertaking, but I’m excited to work on this and see what kind of feedback I get. However, this paper will be more of a long term project, since it will be touching the majority of the wording in the standard. I’ll provide updates in future posts. Placement new during constant evaluation A paper that I’ve been thinking about working on and finally got around to revolves around a new (heh) C++20 feature: new expressions are now able to be evaluated during constant evaluation (CE). While the storage they acquire must be released during the CE and it may only call the replaceable global allocation functions, it finally allows for the use of dynamically sized containers within a constant expression. This is great! However, there is a restriction here that is completely avoidable. The function std::construct_at was introduced to allow for objects to be constructed as if by placement new – nice, but we don’t allow placement new to be used by itself. This is because a certain implementation can’t resolve what object a void* points to during CE (thank you Tim Song for the info); and because CE is intended to always yield the same results on all implementations, construct_at is used to ensure the pointer type passed is always a pointer to object type. I think that at the very least, congruent placement new expressions should be allowed by the principle of this being unnecessarily restrictive. As with all the other papers, I’ll post progress updates in a future post. I’ve drafted up some wording, and I plan to have this ready sometime around June. Information If you want to get in touch with me, you can message me on the Cpplang slack, or shoot me an email.

Continue Reading
03/31/2020
Richard's March Update

Coding in the time of a Pandemic It has been an interesting month, there having been the minor distraction of a lockdown of our little country. The borders with Spain and France were closed about three weeks ago and all residents have been asked to stay at home other than to buy groceries or walk their dogs. Fortunately I have dogs so I at least have a legitimate reason to see the sun. One of the advantages of living in a tiny country is that the government has been able to secure the supply of 150,000 COVID-19 testing kits, which represents two tests per resident. They are also working on supplying every resident with masks for use when shopping. I am hoping to report in my next blog that we are allowed outside subject to a negative test and the wearing of a mask and gloves. Fortunately, until today, our internet has been uninterrupted. Communication with my friends and colleagues at the C++ Alliance and the wider developer community has continued. Boost Release The Boost 1.73 release is imminent. Thus much of my focus in the latter half of the month has been on addressing any remaining issues in Beast that represent an easy win in terms of demonstrating progress between releases. This brings to a close my first quarter as a maintainer of the Beast library. I would have liked to have produced more in terms of feature development and architectural improvements, but a few interesting things came up which delayed this; some of which I will share with you here. (Possibly) Interesting Asio Things To say that Boost.Beast has a strong dependency on Boost.Asio would be an understatement. It should therefore come as no surprise that the Beast team spend a lot of time working with Asio and (certainly in my case) a lot of time working to understand the internals. We had cause to reach out to Chris Kohlhoff, Asio’s author, on two occasions in recent times. If you read my February blog you would have seen the issues we have faced with the DynamicBuffer concept. This month it was about the thread-safety of composed operations and IO objects. But first, the result of a question I asked myself: Is it possible to write an asynchronous composed operation entirely as a lambda? In short, if you’re using c++14 or better, the answer is happily yes! Here is the smallest program I could think of: a: Implemented asynchronously b: Targeting a POSIX system (just because I happen to know more about POSIX than Windows) This program simply copies the contents of stdin to stdout: int main() { asio::io_context ioc; auto exec = ioc.get_executor(); auto in = asio::posix::stream_descriptor(exec, ::dup(STDIN_FILENO)); auto out = asio::posix::stream_descriptor(exec, ::dup(STDOUT_FILENO)); async_copy_all(in, out, [](auto&& ec, auto total){ std::cout << "\ntransferred " << total << " bytes\n"; if (ec.failed()) { std::cerr << "transfer failure: " << ec.message() << std::endl; std::exit(ec.value()); } }); ioc.run(); return 0; } People who are unused to writing composed operations (asynchronous operations that fit into the ASIO ecosystem), or people who have written them longer ago than last year, might at this stage feel their hearts sinking in anticipation of the complex horror show awaiting them when writing the function async_copy_all. Fortunately, Asio’s new(ish) async_compose template function makes this reasonably painless: template<class InStream, class OutStream, class CompletionToken> auto async_copy_all( InStream &fd_in, OutStream &fd_out, CompletionToken &&completion) { return asio::async_compose< CompletionToken, void(system::error_code const &,std::size_t)>( [&fd_in, &fd_out, coro = asio::coroutine(), total = std::size_t(0), store = std::make_unique<char[]>(4096)] (auto &self, system::error_code ec = {}, std::size_t bytes_transferred = 0) mutable { BOOST_ASIO_CORO_REENTER(coro) for(;;) { BOOST_ASIO_CORO_YIELD { auto buf = asio::buffer(store.get(), 4096); fd_in.async_read_some(buf, std::move(self)); } if (ec.failed() || bytes_transferred == 0) { if (ec == asio::error::eof) ec.clear(); return self.complete(ec, total); } BOOST_ASIO_CORO_YIELD { auto buf = asio::buffer(store.get(), bytes_transferred); fd_out.async_write_some(buf, std::move(self)); } total += bytes_transferred; if (ec.failed()) return self.complete(ec, total); } }, completion, fd_in, fd_out); } There are a few things to note in the implementation. The first is that the entire asynchronous operation’s implementation state is captured in the capture block of the lambda (this is why we need c++14 or higher) Secondly, the lambda is mutable. This is so we can update the state and then move it into the completion handler of each internal asynchronous operation. The second and third arguments of the lambda’s function signature are defaulted. This is because async_compose will cause the implementation (in this case, our lambda) to be called once with no arguments (other than self) during initiation. There is an explicit check for eof after the yielding call to fd_in.async_read_some. In Asio, eof is one of a few error codes that represents an informational condition rather than an actual error. Another is connection_aborted, which can occur during an accept operation on a TCP socket. Failing to check for this error-that-is-not-an-error can result in asio-based servers suddenly going quiet for ‘no apparent reason’. Notice that the un-named object created by async_compose intercepts every invocation on it and transfers control to our lambda by prepending a reference to itself to the argument list. The type of Self is actually a specialisation of an asio::detail::composed_op<...> (as at Boost 1.72). However, since this class is in the detail namespace, this should never be relied on in any program or library. Note that I create the buffer object buf in separate statements to the initiations of the async operations on the streams. This is because the unique_ptr called store is going to be moved during the initiating function call. Remember that arguments to function calls are evaluated in unknowable order in c++, so accessing store in the same statement in which the entire completion handler has been moved would result in UB. Finally, async_compose is passed both the input and output stream (in addition to their references being captured in the lambda) so that both streams’ associated executors can be informed that there is outstanding work. It may be surprising to some that the input and output streams may legally be associated with different executors. Actually, now that I write this, it occurs to me that it is unclear to me what is the ‘associated executor’ of the composed operation we just created. Asio’s documentation is silent on the subject. Inspecting the code while single-stepping through a debug build revealed that the executor is taken from the first of the io_objects_or_executors&&... arguments to async_compose which itself has an associated executor. If none of them do, then the system_executor is chosen as the default executor (more on why this may cause surprises and headaches later). Note that as always, wrapping the lambda in a call to bind_executor will force the composed operation’s intermediate invocations to happen on the bound executor. In our case, it is fd_in which will be providing the executor and as a result, every invocation of our lambda (except the first) is guaranteed to be happen by being invoked as if by post(fd_in.get_executor(), <lambda>(...)). system_executor and “What Could Possibly Go Wrong?” Once upon a time, when I first started using Asio, there were no executors at all. In fact, there were no io_contexts either. There was an io_service object. At some point (I don’t remember the exact version of Asio, but it was at least five years ago) the io_service was replace with io_context, an object which did basically the same job. More recently, the io_context represents the shared state of a model of the Executor Named Type Requirement (aka Concept). The state of the art is moving towards passing copies of Executors rather than references to io_contexts. Asio now contains a concrete type, the executor which is a type-erased wrapper which may be assigned any any class which models an Executor. As you might expect, we are heading into a world where there might be more than one model of Executor. In anticipation of this, by default, all Asio IO objects are now associated with the polymorphic wrapper type executor rather than a io_context::executor_type. One such model of Executor supplied by Asio is the system_executor, which is actually chosen as the default associated executor of any completion handler. That is, if you initiate an asynchronous operation in Asio today, against a hypothetical io_object that does not have an associated executor and you do not bind your handler to an executor of your own, then your handler will be invoked as-if by post(asio::system_executor(), <handler>) - that is, it will be called on some implementation-defined thread. Now that the basics are covered, back to what could possibly go wrong? Well imagine a hypothetical home-grown IO Object or AsyncStream. Older versions of the Asio documentation used to include an example user IO Object, the logging socket. The basic premise of our logging socket is that it will do everything a socket will do, plus log the sending and receiving of data, along with the error codes associated with each read or write operation. Clearly the implementation of this object will contain an asio socket object and some kind of logger. The internal state must be touched on every asynchronous operation initiation (to actually initiate the underlying operation and record the event) and during every completion handler invocation, in order to update the logger with the results of the asynchronous operation. As we know, invocations of intermediate completion handlers happen on the executor associated with the final completion handler provided by the user, so in our case, the actions will be something like this: on the initiating thread: logging_socket::async_write_some logging_socket::async_write_some_op::operator()() logging_socket::impl::update_logger(...) socket::async_write_some(...) ... time passes... on a thread associated with the associated executor: logging_socket::async_write_some_op::operator()(ec, bytes_transferred) logging_socket::impl::update_logger() user_completion_handler(ec, bytes_transferred) The situation will be similar for a write operation. Now consider the following code (ls is an object of our hypothetical type logging_socket: ls.async_write_some( get_tx_buffer(), net::bind_executor( net::system_executor(), [](auto ec, auto size){ /* what happens here is not relevant */ })); ls.async_read_some( get_rx_buffer(), net::bind_executor( net::system_executor(), [](auto ec, auto size){ /* what happens here is not relevant */ })); What have I done? Not much, simply initiated a read and a write at the same time - a perfectly normal state of affairs for a socket. The interesting part is that I have bound both asynchronous completion handlers to the system_executor. This means that each of the handlers will be invoked (without synchronisation) on two arbitrary threads. Looking at our pseudo-code above, it becomes clear that there will be a race for the logging_socket’s implementation: Between the initiation of the read and the completion of the write, and between the completion of the read and the completion of the write Again the Asio documentation is silent on the correct method of mitigating this situation. Two possible workarounds have occurred to me so far: Never use a system_executor unless first wrapping it in a strand. Ensure that all composed operations of IO objects are thread-safe with respect to mutation of the implementation. If this is made true, it almost inevitably follows that the entire IO Object may as well be made thread-safe (which Asio IO Objects are not). I have reached out to Chris for final judgement and will update the blog (and possibly much of Beast!) in response to a definitive answer. Unified Web Client I have been given the go-ahead to make a start on exploring a unified web-client library which will eventually become a candidate for inclusion into Boost. The obvious course of action, building directly on top of Beast is a no-go. If the library is to be used on platforms such as tablets and phones, or appear in the various app stores of vendors, there are restrictions on which implementations of communications libraries may be used. To cut a long story short, vendors want to minimise the risk of security vulnerabilities being introduced by people’s home-grown communications and encryption code. So my initial focus will be on establishing an object model that: Provides a high degree of utility (make simple things simple). Emulates or captures the subtleties of vendor’s Web Client frameworks. Efficiently slots into the Asio asynchronous completion model. Of course, linux and proprietary embedded systems do not have a mandated communications libraries, so there will certainly be heavy use of Beast in the unconstrained platform- specific code. More information as it becomes available.

Continue Reading
03/06/2020
Krystian's February Update

Introduction I’m a first-year university student at the University of Connecticut pursuing a degree in Computer Science. I joined The C++ Alliance near the end of January as a part-time Staff Engineer at the recommendation of Vinnie, who has been mentoring me for the past several months. I’ve been programming in C++ for around 2 years now, with an interest in library development and standardization. My original reasoning for choosing C++ over other languages was a little misguided, being that it was generally regarded as a difficult language, and I liked the challenge. Prior to this, I had dabbled in the language a little bit, but I really didn’t get into it until I discovered the existence of the standard. A language with explicitly defined semantics, all specified in a document that is difficult to parse sounded very appealing to me, and thus I embarked on my journey to learn everything about it that I could. If you were to ask me what I like most about it now, it would probably be a tie between the “you don’t pay for what you don’t use” principle and zero cost generics. With regard to standardization, I do a lot of standardese bug-fixing in the form of editorial issues and defect reports, and I write papers mostly focused on fixing the language. As for library development, I’m currently working on Boost.JSON and Boost.StaticString, the latter of which has been accepted into Boost. Boost.JSON My work on Boost.JSON originally started with writing a documentation style guide and then applying it to the library. The first place where this documentation was applied to was json::string; a non-template string type similar to std::string, but with a type erased allocator, used exclusively with char. The documentation of json::string proved to be no easy feat, due to its overwhelming number of overloaded member functions (string::replace itself has 12 overloads). Detailed documentation of these functions did prove to be a little tedious, and once I finished, it was clear that a better interface was needed. The standard interface for std::string is bound to a strict regimen of backward compatibility, and thus the interface suffers due to its lack of the ability to remove stale and unnecessary overloads. Take replace for instance: it has overloads that take a string, a type convertible to string_view, a pointer to a string, a pointer to a string and its size, a string plus parameters to obtain a substring and a string_view plus parameters to obtain a substring. All of these overloads can be replaced with a single overload accepting a string_view parameter, as it is cheap to copy, and can be constructed from all of the aforementioned types. Those pesky overloads taking parameters to perform a substring can also be done away with, as the substring operation can be performed at the call site. json::string includes a subview function which returns a string_view for cheap substring operations, so needless json::string constructions can be avoided. With the guidance of Peter Dimov, I drafted up a new string_view based interface, which dramatically reduced the number of overloads for each member function. Once the sign-off was given, it was a relatively trivial task to implement, as it was merely a matter of removing no longer needed overloads, and changing function templates that accepted objects convertible to string_view to be non-template functions with a string_view parameter. Boost.StaticString I was originally invited by Vinnie to work on Boost.StaticString (then Boost.FixedString) back in October, and since then it has been accepted into Boost. It provides a fixed capacity, dynamically sized string that performs no dynamic allocations. It’s a fast alternative to std::string when either the capacity of the string is known or can be reasonably approximated at compile time. My primary objective for February was to implement the post-review changes requested by our review manager Joaquin Muñoz in time for the next Boost release. Most of the changes by this point had already been implemented, but still lacked polish, and the test coverage had to be improved. Additionally, I wanted to add a few optimizations and improve the constexpr support, the latter proving to be quite a daunting task. The nice thing about static_string is that the capacity of the string is part of the type, consequently allowing us to make all kinds of assumptions and to statically resolve what optimizations we want to perform at compile time. In particular, the usual checks are done in insert and replace that determine if the source of the operation (i.e. the string that will be copied into the string) lies within the string the operation is performed upon can be elided if we can somehow guarantee that they will never overlap. Since non-potentially overlapping objects of different types are guaranteed to occupy distinct addresses, we can safely skip this check for overloads accepting static_string parameters, if the capacity of the strings differ. In my opinion, it’s pretty neat. The aforementioned check that is performed in insert and replace also was the source of some serious headaches with respect to constexpr support across implementations. When you mix in the requirement to support C++11, it gets even worse. The primary issue that presents itself is that comparing pointers with the built-in relational operators yield unspecified results when the pointers do not point to elements of the same array; evaluating such an expression is forbidden during constant evaluation. The usual workaround to this is to use the library comparison operators (i.e. std::less, std::greater etc.), and in standardese land, it’s all good and well. However, on actual implementations, this won’t always work. For example, when using clang with libstdc++, the implementation for the library comparison operators casts the pointers to integer types and compares the resulting integers. If __builtin_is_constant_evaluated is available it is used and the check is instead done using the built-in relational operators, but this was only implemented on clang quite recently and therefore cannot be used for the most part. Figuring this out and finding a solution was quite a process, but eventually, I settled on a solution that gave the least unspecified results across implementations, provided good performance, and actually compiled. In essence, if std::is_constant_evaluated or a builtin equivalent is available, we can use a neat trick involving the equality operator for pointers. Equality comparison for pointers almost never has unspecified results, so to check if the pointer falls within a range, we can iterate over every pointer value within that range and test for equality. This is only done during constant evaluation, so performance does not suffer. If the aforementioned functions cannot be used, or if the function is not evaluated within a constant evaluation, we use the built-in comparison operators for configurations where the library comparison operators don’t work in constant evaluation, and otherwise use the library comparison operators. Having a portable version of this check in the standard library wouldn’t be a terrible idea, so it may be something I will pursue in the future. Another feature that took a little bit of time to get working were the to_static_string overloads for floating-point types. static_string does not enjoy the luxury of being able to increase capacity, so sticking to only using standard notation isn’t possible since floating-point types can represent extremely large values. To get the best of both worlds, we first attempt to use standard form and then retry using scientific notation if that fails. To match std::to_string, the default precision of 6 is used for standard form - however, if we resort to scientific notation, we use the highest precision value possible based on the number of digits required to represent all the values of the type being converted, and the number of digits in its maximum mantissa value. As Peter Dimov noted several times, using the %g format specifier would be a preferable solution, so I may change this in the future. Standardization The Prague meeting came and went (I did not attend), and unfortunately no progress was made on any of my papers. CWG was very busy finalizing C++20, so that left no time for a wording review of P1839, and I’m holding off on my other papers. I’m planning to attend the New York meeting in November representing The C++ Alliance (my first meeting, I’m excited!), and is where I will be presenting P1945, P1997, and a few others that are currently in the works. Outside of proposals, it was business as usual; finding and fixing several editorial issues, and filing a few defect reports. Most of the work I do focuses on fixing the core language, and generally improving the consistency of the wording, so this month I worked on fixing incorrect conventions, removing redundant wording, and a small rewrite of [dcl.meaning] on the editorial side of things. As for defect reports, they consisted of some small wording issues that weren’t quite editorial but didn’t have a significant impact on the language. Summary My experience working at the Alliance has been very positive thus far. Being a student, having flexible hours is fantastic, as I am able to adjust when I work based on my school workload. In the short amount of time I have spent working on Boost.JSON and Boost.StaticString, I have learned a lot, and continue to do so every day. Vinnie, Peter, and Glen always provide their invaluable feedback through reviews and answering questions, which is extremely helpful when working on projects of this scale with little experience. I consider the acceptance of Boost.StaticString into Boost to be my crowning achievement thus far, and I’m excited to see what kinds of cool projects I’ll be working on in the future. If you want to get in touch with me, you can message me on the Cpplang slack, or shoot me an email.

Continue Reading
02/29/2020
Richard's February Update

The Lesser HTTP Refactor Aside from the normal answering of queries and issues, February has been a month dominated by the difference between the asio::DynamicBuffer_v1 and asio::DynamicBuffer_v2 concepts. As I understand things, both Beast and Asio libraries developed the idea of the DynamicBuffer concept (or more correctly, Named Type Requirement [NTR]) at roughly the same time, but with slightly different needs. The original Asio DyanmicBuffer describes a move-only type, designed to be a short-lived wrapper over storage which would allow a composed operation to easily manage data insertions or retrievals from that storage through models of the MutableBufferSequence and ConstBufferSequence NTRs. In Beast, it was found that DynamicBuffer objects being move-only caused a difficultly, because the necessarily complex composed operations in Beast need to create a DynamicBuffer, perform operations on it, pass it to a sub-operation for further manipulation and then continue performing operations on the same buffer. If the DynamicBuffer as been passed by move to a sub operation, then before the buffer can be used again, it will have to be moved back to the caller by the callee. Rather than complicate algorithms, Beast’s authors took a slightly different track - Beast DynamicBuffers were specified to be pass-by-reference. That is, the caller is responsible for the lifetime of the DynamicBuffer and the callee is passed a reference. This satisfied Beast’s needs but created an incompatibility with Asio and Net.TS. Vinnie Falco wrote a paper on the problem offering a solution to enabling complex composed operations involving DynamicBuffers. On reflection, LEWG took a different view and solved the problem by re-engineering the DynamicBuffer NTR. The result is that Boost.Beast objects are now likely to encounter three versions of DynamicBuffer objects in the wild and needs to be able to cope gracefully. Boost.Asio now has the NTRs DynamicBuffer_v1 and DynamicBuffer_v2, with the NTR DynamicBuffer being a synonym for either depending on configuration flags (defaulting to DynamicBuffer_v2). We have had to go a little further and add a new NTR in Beast, DynamicBuffer_v0. The meanings of these NTRs are: NTR Mapping in Asio Mapping in previous Beast Overview DynamicBuffer_v0 none DynamicBuffer A dynamic buffer with a version 1 interface which must be passed by reference DynamicBuffer_v1 DynamicBuffer_v1 Asio DynamicBuffer (v1) A dynamic buffer with a version 1 interface which must be passed by move DynamicBuffer_v2 DynamicBuffer_v2 none A dynamic buffer with a version 2 interface which is passed by copy My intention this month was to migrate the entire Beast code base to use Asio’s (and current net.ts’s) DynamicBuffer_v2 concepts while still remaining fully compatible with DynamicBuffer_v0 objects (which will be in existing user code). The first attempt sought to change as little of the Beast code as possible, by writing DynamicBuffer_v0 wrappers for DynamicBuffer_v[1|2] objects, with those wrappers automatically created on demand in the initiating function of Beast IO operations. The problem with this approach is that it penalised the use of DynamicBuffer_v2 with an additional memory allocation (in order to manage a proxy of DynamicBuffer_v1’s input and output regions). On reflection, it became apparent that in the future, use of DynamicBuffer_v2 will be the norm, so it would be inappropriate for Beast to punish its use. Therefore, we have chosen to take the harder road of refactoring Beast to use DynamicBuffer_v2 in all composed operations involving dynamic buffers, and refactor the existing DynamicBuffer_v0 types in order to allow them to act as storage providers for DynamicBuffer_v2 proxies while still retaining their DynamicBuffer_v0 public interfaces. I had hoped to get all this done during February, but alas - in terms of released code - I only got as far as the refactoring of existing types. The code has been released into the develop branch as part of Version 286. The refactor of HTTP operations and Websocket Streams in terms of DynamicBuffer_v2 is underway and indeed mostly complete, but there was not sufficient time to release a sufficiently robust, reviewed and tested version this month. I plan to finish off this work in the first part of March, which will hopefully leave time for more interesting challenges ahead. What’s Next Well, the Beast Issue Tracker is far from clear, so there is plenty of work to do. At some point though, I’d like to make a start on a fully featured higher level HTTP client library built on Boost.Beast. We’ll have to see what unfolds.

Continue Reading
01/31/2020
Richard's January Update

History This is my first entry on the C++ Alliance web site. I’m very happy to say that I was invited to join the organisation at the end of December last year. I first met Vinnie on Slack when I chose to use Boost.Beast in a greenfield project - a highly scalable market data distribution system and quoting gateway for the Japanese cryptocurrency exchange liquid.com. There were a number of candidates for C++ HTTP frameworks and it is interesting for me to examine the decision-making process I went through in choosing one. If I am honest, there are two main factors that influenced me towards Boost.Beast: I am a long-time fanboi of Boost.Asio. I find it’s paradigm very pleasing. Once you decipher the (extremely terse!) documentation it becomes obvious that it was written by a hyper-intelligent extraterrestrial masquerading as a human being. I have used the Boost Library (or more correctly, libraries) for many years. Boost has become synonymous with trust, quality and dependability. As far as I have always been concerned, boost is the standard. The standard library has always been a pale shadow of it. When I started the new project there was an expectation that I would have a team to work with. In the end, I found myself writing the entire system alone from scratch. Liquid Tap contains two fully-featured web servers (one facing the organisation and one facing the outside world), supports inbound and outbound websocket connectivity (with per-fqdn keepalive connection pooling) and multi-threaded operation. The project took me 3 months from writing the first line of code to full production readiness. I was personally impressed by the speed with which I was able to assimilate a new library and create a fairly complex service infrastructure using nothing more than boost, nlohmann_json, openssl and a c++17 compiler. In my view this would not have been possible without the excellent documentation and careful attention to detail in Boost.Beast. During the development, I reached out to Vinnie and Damian on Slack a number of times. Both were always helpful and attentive. Without a doubt they were instrumental in the success of my project. So here I am in January 2020. Just like the old TV advert where Victor Kayam was so impressed with his electric shaver that, “I bought the company”. I was so impressed with Boost.Beast and its creators that when given the chance, I chose to join the company. First Month I have spent my first month with the firm going through Vinnie’s Boot Camp. It’s been an interesting and invigorating experience. I have many years of experience writing code for production environments, from embedded systems like slot machines and video games to defence to banking and trading systems. I’m fairly confident that if you can describe it, I can write it. But maintaining a publicly-facing library is a new and very different challenge. Controlling the Maintenance Burden C++ is a language in which types are cheap. So cheap in fact that many people (including me) recommend describing any concept in a program as its own type. This is a great way to cause logic errors to fail to compile, rather than fail to impress your customers. But in a library such as Boost.Beast, every public type you create is a type you must document and maintain. If you discover that your type has a design flaw, you’re a little stuck. Any future changes need to be source-compatible with previous versions of the library. Dangerous or incorrect elements in the design must be deprecated gracefully. All this requires careful management. Management takes time. Something I learned from Vinnie very quickly is that for this reason, interfaces to public objects should provide the bare minimum functionality that users can demonstrate a need for. Adding a new public method or class should only happen after careful consideration of the consequences. Supporting all Toolchains Another consideration I have never had to encounter before is that Boost.Beast is designed to work on every compiler that “robustly supports” C++11. In my career as a software engineer I have always demanded (and mostly had) autonomy over the choice of toolset. Of course I have always chosen the very latest versions of gcc, clang and msvc and used the very latest standard of c++ available. Why wouldn’t I? It improves productivity, and who wants to be stuck in the Dark Ages when all your friends are using the new cool stuff? I wrote Liquid Tap in C++17. If C++2a had been more advanced and supported by all compilers at the time I’d have used that, because compiler-supported coroutines would have shortened the code and made debugging and reasoning about sequencing a whole lot easier. Now I find myself thinking about how to support the latest features of the language, while ensuring that the library will continue to function for the many C++11 and 14 users who have not been as fortunate as me and are still constrained by company policy, or indeed are simply happy not to have to learn the new features available in more recent standards. Attention to Detail Boost.Beast strives for 100% (or as close to it as possble) coverage in testing. This is only logical. Users are not going to be happy if they have to continually decipher bugs in their programs caused by us, file bug reports and either patch their Boost source or wait up to three months for another release. In addition, documentation matters. You know how it is in a production system. More effort is spent on content and utility than documentation. Developers are often expected to read the code or ask someone if there’s something they don’t understand. Not so when maintaining a library for public consumption. One of the reasons I chose Boost.Beast was the quality, completeness and accuracy of its documentation. This is no accident. Vinnie and his team have put a lot of time into it. Only now am I becoming aware of what an Herculean task this is. Users will hold you to your word, so your word had better be the Truth, the Whole Truth and Nothing But the Truth. Activities Issue Maintenance This month I have been working through some of the Issue backlog in Boost.Beast. It’s satisfying to see PRs getting accepted and the list of open issues being whittled away. At the moment it’s interesting to see new issues and queries being raised too. I’ll revisit this in next month’s blog and report as to whether it’s still interesting then :) I have also been taking time to liaise with users of the library when they raise queries via the Issue Tracker, email or the Slack Channel. I think staying in touch with the users is an excellent way to get feedback on the quality of documentation and design. It’s also nice to be able to help people. Not something you get time to do very often when working on an FX-options desk in an investment bank. Boost.Json I have been providing some support to the upcoming Boost.JSON library. This library focusses on: Absolute correctness in reference to RFC8259. Seeking to match or exceed the performance of other c++ JSON libraries such as RapidJSON. Providing a clean, unsurprising programming interface and impeccable documentation. This is a fascinating project for me. Various JSON libraries employ various tricks for improving performance. Performance can be gained at the expense of rigorous syntax checking, use of buffers and so on. Achieving the Holy Grail of full correctness, minimal state and absolute performance will be an interesting challenge. Boost.URL Vinnie is also working on Boost.URL. While I have not contributed any code, spending my time in the #beast Slack channel has meant that I’ve been able to keep up to speed with the various design choices being made. Again, there has been much to learn about a design rationale that focuses heavily on the potential maintenance burden. There is actually a lot that could be learned by developers in industry from taking part in or observing this discourse. Work Schedule The C++ Alliance is based on the West Coast of the USA, while I live and work in the tiny Principality of Andorra in mainland Europe. This puts me some nine hours ahead of my colleagues across the Pond. It turns out that this is a perfect way of working for me. I can get up at 8am, nip out for a couple of hours skiing or hiking, enjoy lunch and then get to work - before anyone else in the firm is even awake. I’m a bit of a night-owl anyway, so working later in order to engage in “lively debate” with my colleagues on Slack is no problem. It also means I have a legitmate excuse to get out of any social engagments I don’t want to be bothered with. So for me it’s all win. Summary I’ve really enjoyed my first month. I think Vinnie worries that he’ll nag me too much about seemingly unimportant details like commit message wording and achieving a certain tone in code documentation, but I don’t mind it. Boost.Beast is a fantastic piece of work. It’s Vinnie’s baby, and I am privileged to be asked to hold it in my hands. I’m never going to take issue with a mother looking to protect her cubs. Furthermore, having a legitimate excuse to interact with the other maintainers of Boost on Slack is a pleasure. These people are some of the brightest minds on the planet. I live in hope that some of this brilliance will rub off. If you work with C++, I highly recommend that you join the Cpplang Slack channel. If you’d like to contact me to discuss my experiences I’d be happy to receive a message on Slack. Thanks for reading. Richard Hodges

Continue Reading
01/30/2020
Gold sponsor of C++Now 2020

The Alliance is a Gold sponsor for C++Now 2020. This conference is a gathering of C++ experts and enthusiasts from around the world in beautiful Aspen, Colorado from May 3, 2020 - May 8, 2020.

Continue Reading
09/27/2019
Marshall's Combined August and September Update

There are four main areas where I spend my time. Libc++, where I am the “code owner” WG21, where I am the chair of the Library Working Group (LWG) Boost Speaking at conferences Lots of work these month(s) behind the scenes, getting stuff ready for C++20, LLVM 9, and Boost 1.71.0. Libc++ The LLVM 9.0 release has shipped! The release date was 19-September, a few days later than planned. There are a lot of new libc++ features in the release. As the “code owner” for libc++, I also have to review the contributions of other people to libc++, and evaluate and fix bugs that are reported. That’s a never-ending task; there are new contributions ever day. Many times, bug reports are based on misunderstandings, but require a couple of hours of work in order to figure out where the misunderstanding lies. We’re working on a major redesign of the “debug mode” for libc++, after we realized that the existing (not widely used) debug mode is useless when you’re trying to do things at compile (constexpr) time. I have been spending a lot of time the last few weeks working on the calendaring stuff in <chrono>, specifically the interface with the OS for getting time zone information. It is a surprisingly complicated task. Fortunately for me, I have a friend who has been down this road in the past, and is willing to answer questions. LWG issues resolved in libc++ (almost certainly incomplete) LWG3296 Add a missing default parameter to regex::assign LLVM features implemented (almost certainly incomplete) P1466 Parts of P1466 “Misc Chrono fixes” more to come here. LLVM bugs resolved (definitely incomplete) Bug 42918 Fix thread comparison by making sure we never pass our special ‘not a thread’ value to the underlying implementation Bug 43063 Fix a couple of unguarded operator, calls in algorithm Bug 43034 Add a missing _VSTD:: before a call to merge. Bug 43300 Add a missing _VSTD:: Only initialize the streams cout/wcout/cerr/wcerr etc once, rather than any time Init::Init is called Other interesting LLVM bits from (certainly incomplete) Revision 368299 Implement hh_mm_ss from P1466. Part of the ongoing <chrono> implementation work. The current status of libc++ can be found here: C++20 status C++17 status C++14 status (Complete) Libc++ open bugs WG21 We shipped a CD out of Cologne in July. Now we wait for the National Bodies (members of ISO, aka “NBs”) to review the draft and send us comments. When we’ve resolved all of these comments, we will send the revised draft out for balloting. If the NBs approve, then that draft will become C++20. The next WG21 meeting will be November 2-8 in Belfast, Northern Ireland. This will be the first of two meetings that are focused on resolving NB comments; the second one will be in Prague in February. I have several “clean-up” papers for the Belfast mailing. The mailing deadline is a week from Monday (5-October), so I need to finish them up. P1718R0: Mandating the Standard Library: Clause 25 - Algorithms library P1719R0: Mandating the Standard Library: Clause 26 - Numerics library P1720R0: Mandating the Standard Library: Clause 28 - Localization library P1721R0: Mandating the Standard Library: Clause 29 - Input/Output library P1722R0: Mandating the Standard Library: Clause 30 - Regular Expression library P1723R0: Mandating the Standard Library: Clause 31 - Atomics library We polled the NBs before Cologne, and they graciously agreed to have these changes made post-CD. Boost Boost 1.71.0 was released on 19-August. Micheal Caisse was the release manager, with some help from me. As part of the Boost Community maintenance team, I (and others) made many changes to libraries whose authors are no longer able (or interested) in maintaining them. I have a couple of suggestions for additions to the Boost.Algorithms library that I will be working on in the near future. Conferences I was a speaker at CppCon last week. I gave a new talk “std::midpoint - How hard could it be?” (no link yet) which was quite well received. I got a few questions that will require additional research, and may improve my implementation. I also participated in the “Committee Fireside Chat”, at CppCon, where conference members get to ask questions of the committee members who are present. Upcoming talks: LLVM Developer’s Conference is in San Jose in October. I will not be speaking, but I will be moderating the lightning talks. C++ Russia is at the end of October in St. Petersburg. ACCU Autumn is right after the WG21 meeting in early November. Meeting C++ is in mid-November in Berlin. I will be making the “Fall 2019 C++ European Tour”, going from St. Petersburg to Belfast to Berlin before heading home mid-November.

Continue Reading
09/01/2019
Gold Sponsor Of Cppcon 2019

The Alliance is a Gold sponsor for CppCon 2019. This conference is the annual, week-long face-to-face gathering for the entire C++ community. The conference is organized by the C++ community for the community. Attendees enjoy inspirational talks and a friendly atmosphere designed to help individuals learn from each other, meet interesting people, and generally have a stimulating experience.

Continue Reading
08/19/2019
Damian's July Update

Boost.Beast I’ve started working on improvements to the zlib part of Beast. There are some gaps in the test harness of these components, so I’ve decided to increase coverage. As a first step, I started porting test cases from the original zlib library’s tests, to verify that existing code matches the expected behavior of the original library. Fortunately, I’ve not found any significant discrepancies, there’s only one issue where Beast rejects malformed input for the wrong reason (I’m still looking into it whether it’s actually an issue at the time of writing this). I’ve also looked into providing more meaningful feedback from test failures in Beast, especially when they’re run in CI. While the current test framework does print a line number on failure, the line number is often in a function template that’s called by multiple test cases, which makes it quite hard to determine which test failed just from the log, often requiring the use of a debugger. Doing that locally may not be a problem, but it’s significantly harder in CI, so I’ve decided to try to use Boost Stacktrace to provide a callstack on each failure in Beast tests. Additionally, I’ve also worked on running the test suite without OpenSSL installed, to hopefully fix some of the failures in the official Boost test matrix. The question of Networking TS and TLS There’s recently been quite a bit of discussion of networking being useless without “secure by default” sockets. Since this is a recurring topic and I expect it to return in the future, so I’ve decided to write up an analysis of this issue. First of all, I believe that an attempt to deliver a “secure by default” socket within the current networking proposal right now will result in something like std::async - not really practically useful. What kind of TLS facilities I’d consider useful for the average user of the standard library? A reasonable guideline, I think, are ones I could trust to be used in a distributed system that handles money (in any form). Note, that TLS is not only a protocol that provides confidentiality (i.e. encryption), but also allows verification of the identity either the server by the client, or both. Remember, doesn’t matter if 3rd parties can’t see what you’re sending, if you’re sending your data to the wrong peer in the first place! While it may seem simple at first look, just verifying the identity of a peer is an extremely complex process, as my experience with Certify has shown. Doing it portably and efficiently with the same interface and effects is extremely difficult. Browsers resort to all kinds of workarounds and custom solutions to be able to securely implement just this one aspect of TLS. I attempted to implement a library (intended for inclusion into Boost) that would perform this one aspect, however, I found it to be impossible to provide a practical solution with the current state of the networking ecosystem in Boost. In fact, one method of certificate verification (via the OCSP protocol) requires a (very) basic HTTP client. Yes, in order to perform a TLS handshake and verify the peer’s certificate status using OCSP, you need an HTTP client. This is just one aspect of the TLS protocol that needs to be addressed. There are others as well - what about the basic cryptographic building blocks, like ciphers, hashing algorithms, PRFs and so on - they are bound to be used in a hypothetical implementation in a standard library, should they be exposed? If yes then with what interface?. Considering that there are no standard networking facilities and not even a proposal for standard TLS, this is a discussion that would essentially postpone standard networking indefinitely. Finally, there’s also an opposite position that no networking should be in the standard at all. I disagree with this position - networking has become a very important part of many C++ projects (in my career, all C++ projects I dealt with, touched some sort of network in one way or another). At the very least we need standard named requirements for library compatibility, since that is severely lacking in the ecosystem at this point.

Continue Reading
08/05/2019
Marshall's July Update

There are four main areas where I spend my time. Libc++, where I am the “code owner” WG21, where I am the chair of the Library Working Group (LWG) Boost Speaking at conferences This month, the big news (and the big work item) was the approval of the C++ “Committee Draft” at the WG21 meeting in Cologne on July 15-20. You can think of this as a “beta version” of the C++20 standard; all features are complete. The next step is bug fixing, with an eye towards releasing next year. Libc++ The LLVM 9.0 release is on track for September. We have a release branch, and the RC1 was recently dropped. Because of the run-up and the aftermath of the Cologne meeting, the libc++ accomplishments are a bit sparse this month. As the “code owner” for libc++, I also have to review the contributions of other people to libc++, and evaluate and fix bugs that are reported. That’s a never-ending task; there are new contributions ever day. LWG issues resolved this month in libc++ (almost certainly incomplete) LWG2273 regex_match ambiguity LLVM features implemented this month (almost certainly incomplete) P1612 Relocate endian P1466 Parts of P1466 “Misc Chrono fixes” more to come here. LLVM bugs resolved this month (definitely incomplete) Other interesting LLVM bits from this month (certainly incomplete) Revision 365854 Reorganize the <bit> header to make most of the facilities available for internal use pre-C++20. NFC for external users. Revision 367120 Fix a bug in std::chrono::abs where it would fail when the duration’s period had not been reduced. Revision 364884 Add an internal call __libcpp_is_constant_evaluated, which works like std::is_constant_evaluated, except that it can be called at any language level (back to C++98). For older languages, it just returns false. This gets rid of a lot of ifdefs in the libc++ source code. The current status of libc++ can be found here: C++20 status C++17 status C++14 status (Complete) Libc++ open bugs WG21 As I said above, we shipped a CD out of Cologne. Now we wait for the National Bodies (members of ISO, aka “NBs”) to review the draft and send us comments. When we’ve resolved all of these comments, we will send the revised draft out for balloting. If the NBs approve, then that draft will become C++20. We approved many new features for C++20 in Cologne: P0553 - Bit Operations P0980 - Constexpr string P1004 - Constexpr vector P1065 - Constexpr INVOKE P1135 - The C++20 Synchronization Library P1208 - Source Location P0645 - Text Formatting P1361 - Integration of chrono with text formatting P1754 - Rename concepts to standard_case for C++20, while we still can P1614 - Spaceship integration in the Standard Library P0600 - Stop Tokens and a Joining Thread P0631 - Math Constants We also did not approve many proposed features. Most of these were not approved because we ran out of time, rather than any fault of theirs: P1391 - Range constructors for string_view P1394 - Range constructors for span P0288 - any_invokable P0201 - polymorphic_value P0429 - A Standard flatmap P1222 - A Standard flatset P0533 - constexpr for cmath P0792 - function_ref P0881 - A Proposal to add stacktrace library P1272 - Byte-swapping P0627 - Function to mark unreachable code and many others I still have a bunch of mechanical changes that need to be made before we ship: P1718R0: Mandating the Standard Library: Clause 25 - Algorithms library P1719R0: Mandating the Standard Library: Clause 26 - Numerics library P1720R0: Mandating the Standard Library: Clause 28 - Localization library P1721R0: Mandating the Standard Library: Clause 29 - Input/Output library P1722R0: Mandating the Standard Library: Clause 30 - Regular Expression library P1723R0: Mandating the Standard Library: Clause 31 - Atomics library We polled the NBs before Cologne, and they graciously agreed to have these changes made post-CD. Boost The next Boost release cycle is in process; I am helping Michael Caisse as release manager with this release. We should have a release in the next couple of weeks. Conferences Upcoming talks: CppCon in September in Denver. C++ Russia is at the end of October in St. Petersburg. ACCU Autumn is right after the WG21 meeting in early November. Meeting C++ is in mid-November in Berlin. I will be making the “Fall 2019 C++ European Tour”, going from St. Petersburg to Belfast to Berlin before heading home mid-November.

Continue Reading
07/14/2019
Damian's June Update

This month I’ve been working on the following projects: Certify Boost.Beast Certify After quite a bit of work exploring ways to make certificate verification more complete, I’ve concluded that Boost is currently missing a few tools to make that viable. A comprehensive solution requires, at the very least, a functional HTTP client able to handle higher-level semantics like redirects, proxies or compressed bodies. While these are unlikely to happen while performing an OCSP query or downloading a CRL set from Google’s update service, they still need to be handled, otherwise the user will be left in a no better state than when no library is used. At this point, Certify only offers basic verification, but that is still simillar level to what cURL does. Providing a comprehensive solution will require either a infrastructure solution (something like Google’s CRLsets) or a library based one (i.e. build up all the required libraries to be able to perform proper certificate status checks). Boost.Beast I’ve continued the work on expanding split compilation in Beast, by turning some internal function templates, in websocket code, into regular functions. Additionally, I’ve simplified the websocket prng code after proving with some benchmarks that the previous solution made it worse both for the fast case (with TLS enabled) and the slow one. The speed up is marginal, but it made the code much simpler and reduced the size of binaries by small amount (a few K at best). I’ve also worked to cleaning up some of the compilation warnings that I found using the new Azure Piepelines CI in Beast. I also had to deal with an an odd case of miscompilation under MSVC 14.2 (x64 Release), where the use of static_string<7> failed tests with paritally garbage output while static_string<8> succeeded.

Continue Reading
07/02/2019
Marshall's June Update

There are four main areas where I spend my time. Libc++, where I am the “code owner” WG21, where I am the chair of the Library Working Group (LWG) Boost Speaking at conferences Libc++ The next big milestone for libc++ is the LLVM 9.0 release this summer. We’re working towards that, implementing new features and fixing bugs. The “Branch for release” is currently scheduled for July 18th. As the “code owner” for libc++, I also have to review the contributions of other people to libc++, and evaluate and fix bugs that are reported. That’s a never-ending task; there are new contributions ever day. I created a status page for the LWG issues and papers that are already set up for a vote at the Cologne WG21 meeting. LWG issues resolved this month in libc++ (almost certainly incomplete) LWG2221 No formatted output operator for nullptr LWG3206 year_month_day conversion to sys_days uses not-existing member function LLVM features implemented this month (almost certainly incomplete) P0553 Bit operations P0556 Integral power-of-2 operations P1355 Exposing a narrow contract for ceil2 P0646 Improving the Return Value of Erase-Like Algorithms I LLVM bugs resolved this month (probably incomplete) Bug 41843 std::is_base_of should give correct result for incomplete unions Bug 38638 Wrong constraint for std::optional<T>::operator=(U&&) Bug 30589 std::complex with a custom type does not work because of how std::__promote is defined Bug 42396 Alignment not respected in containers for over-aligned enumeration types Bug 28704 num_get::do_get incorrect digit grouping check Bug 18074 Undefined references when using pointer to member functions Bug 26503 std::quoted doesn’t work with char16_t or char32_t strings. Bug 41714 std::tuple<> is not trivially constructible Bug 36863 basic_string_view(const CharT*, size_type) constructor shouldn’t comment out assert of nullptr and length checks Bug 42166 to_chars can puts leading zeros on numbers Other LLVM bits from this month (certainly incomplete) Revision 364545 Provide hashers for string_view only if they are using the default char_traits. Reported on StackOverflow Reworked to_string to use to_chars. Much faster, and avoids having multiple implementations. This involved reworking to_chars so that it was available back to C++03. I did all of the to_chars refactoring, but not the to_string rework. The current status of libc++ can be found here: C++20 status C++17 status C++14 status (Complete) Libc++ open bugs WG21 The next WG21 meeting is July 15-20 in Cologne, Germany. There were no WG21 meetings in June. We (LWG) held four teleconference this month, reviewing papers in advance of the July meeting, and will have another one next week. I had seven papers in the pre-Cologne mailing: P1718R0: Mandating the Standard Library: Clause 25 - Algorithms library P1719R0: Mandating the Standard Library: Clause 26 - Numerics library P1720R0: Mandating the Standard Library: Clause 28 - Localization library P1721R0: Mandating the Standard Library: Clause 29 - Input/Output library P1722R0: Mandating the Standard Library: Clause 30 - Regular Expression library P1723R0: Mandating the Standard Library: Clause 31 - Atomics library P1724R0: C++ Standard Library Issues to be moved in Cologne The goal of the July meeting is to have a “Committee Draft” (CD) of the proposed C++20 standard that can be sent out for review. Also on my TODO list is to attempt to implement some of the proposals that are coming up for a vote in July (flat_map, text formatting, etc). Boost The next Boost release cycle is in process; I am helping Michael Caisse as release manager with this release. Conferences Upcoming talks: C++ Russia is at the end of October in St. Petersburg. Meeting C++ is in mid-November in Berlin. I have submitted a talk for CppCon in September, but I will not hear back about this for a month or two. I submitted a talk for ACCU Autumn, which is in Belfast right after the WG21 meeting, but I haven’t heard back about that yet. In any case, I will be attending this conference, since it’s in the same hotel as the WG21 meeting, and starts two days after the WG21 meeting, and concludes right before Meeting C++.

Continue Reading
06/10/2019
Damian's May Update

This month I’ve been working on the following projects: Certify Boost.Beast Boost.Build Certify This month, I’ve worked on expanding the documentation of Certify, especially the example and introduction parts. When looking through the documentation for Boost.Build I found out it’s possible to import snippets from *.cpp files into the documentation, which will allow me to make sure that snippets in the documentation compile and are tested. I’ve also attempted cleaning up the Certify build script to use the OpenSSL module in b2, but I ran into issues, so I’ll have get back to this one in the future. Don’t forget to star the repository: https://github.com/djarek/certify! Boost.Beast I’ve been able to complete the port of the Beast CI to Azure Pipelines and expand the test matrix beyond what was tested in the existing CI infrastructure. Thanks to the expanded concurrent job limit, a full run on AzP takes less time than a full Travis and Appveyor build, especially when wait times are taken into accout. One of the matrix items I added were tests for header-only no-deprecated builds, which turned out to be broken. Untested code has a nasty tendency to rot. I’ve also been able to identify some function templates in http::basic_fields which could be turned into regular functions. One of them, was instantiated 4 times because they were passed a predicate which was a lambda expression. These two changes turned out to be fairly significant, because they allowed shaving off at least 10 KiB of binary size per instantiation (amd64, -O3). Boost.Build When working on the Azure Pipelines CI for Beast I noticed that b2 doesn’t support the leak sanitizer, so I decided to add it. It’s available via the leak-sanitizer=on feature.

Continue Reading
06/01/2019
Marshall's May Update

There are four main areas where I spend my time. Libc++, where I am the “code owner” WG21, where I am the chair of the Library Working Group (LWG) Boost Speaking at conferences Libc++ The next big milestone for libc++ is the LLVM 9.0 release this summer. We’re working towards that, implementing new features and fixing bugs. As the “code owner” for libc++, I also have to review the contributions of other people to libc++, and evaluate and fix bugs that are reported. That’s a never-ending task; there are new contributions ever day. This month was spent concentrating on code reviews and bug reports; so I implemented very little “new code”. There was a lot of “infrastructure work” done on libc++ this month; a large cleanup of the test suite (still in progress), a bunch of work on debug mode for the library (also still in progress) LWG issues resolved this month in libc++ 2960 sub_match::swap only swaps the base class LLVM features implemented this month (certainly incomplete) Improved the behavior of the compiler intrinsic __is_base_of. Clang no longer generates an error when you ask about inheritance relationships with unions, even if the non-union class is incomplete. This intrinsic is used by libc++ to implement std::is_base_of. Fixed a few regex bugs, and improved the regex tests in C++03. LLVM bugs resolved this month (probably incomplete) Bug 42037 C++2a std::midpoint`’s “Constraints” are not implemented Bug 41876 std::hash should not accept std::basic_strings with custom character traits The current status of libc++ can be found here: C++20 status C++17 status C++14 status (Complete) Libc++ open bugs WG21 There were no WG21 meetings in April. However, LWG held one teleconference this month, reviewing papers in advance of the July meeting. We’ll have more teleconferences in June. I am working on more “cleanup” papers similar to P1458 - Mandating the Standard Library: Clause 16 - Language support library, and my P0805 - Comparing Containers needs an update. The goal of the July meeting is to have a “Committee Draft” (CD) of the proposed C++20 standard that can be sent out for review. Also on my TODO list is to attempt to implement some of the proposals that are coming up for a vote in July (flat_map, text formatting, etc). Boost It’s been a quiet month for boost (except for C++ Now, the conference formerly known as BoostCon). There are a couple of good trip reports for C++Now: Matthew Butler JeanHeyd Meneide The next Boost release cycle is starting soon; with the deadline for new libraries coming up later this month. I’m hoping to mentor a new release manager with this release. Conferences Another travel month. I spent a bunch of time away from home, but only one conference: At C++ Now in Aspen, CO, I presented “The View from a C++ Standard Library Implementor”, which was voted the runner-up for “Most Engaging” talk. I have submitted a talk for CppCon in September, but I will not hear back about this for a month or two. I have submitted talks for C++ Russia and Meeting C++, which are both very close (timewise) to the Belfast WG21 meeting, but I haven’t heard back yet. I am looking forward to being at home for the entire month of June.

Continue Reading
05/05/2019
Damian's April Update

This month I’ve been working on the following projects: Certify Boost.Beast Boost.Build BeastLounge Certify Certify did not have any platform-independent means of caching certificate status (i.e. revoked, valid, unknown), so I implemented one. For now it has to be manually filled, but I’ll add a way to import a static blacklist (somewhat similar to the builtin blacklist in Chrome) and query the status of a certificate. Unfortunately there is no way to handle OCSP stapling within the verification callback invoked by OpenSSL which is quite detrimental to usability. Additionally, OpenSSL doesn’t have a way of starting and waiting for an asynchronous operation within callbacks (without blocking). Don’t forget to star the repository: https://github.com/djarek/certify! Boost.Beast When working on making sure Beast is std::launder-correct, I ran into a number of previously undiagnosed bugs in Beast. All of them have been fixed in v254. I was quite confused why these issues weren’t found by CI previously. I’ve been able to track it down to old toolchain versions in Travis. Additionally, the test matrix lacks a few fairly important variants. Considering the fact that Trusty is no longer supported and the switch to Xenial is inevitable, I’ve decided to port over the CI to Azure Pipelines, because it offers better concurrency which allows the Beast CI to afford a larger test matrix. In the process I’ve also decided to use as many default b2 options as possible, to make future changes to the CI easier. There’s still an issue with Valgrind in Xenial to be resolved (doesn’t support the RDRAND instruction). Boost.Build While working on the AzP CI for Beast, I found out that the coverage feature in b2 doesn’t actually set build flags. coverage=all will now properly cause tests to produce gcno and gcda files for consumption by the lcov tool. BeastLounge When experimenting with the BeastLounge application running on Heroku I found out that Heroku’s router has a builtin 55s timeout which dropped websocket connections. I solved the issue by making the websocket ping timeouts configurable.

Continue Reading
05/01/2019
Marshall's April Update

There are four main areas where I spend my time. Libc++, where I am the “code owner” WG21, where I am the chair of the Library Working Group (LWG) Boost Speaking at conferences Libc++ The next big milestone for libc++ is the LLVM 9.0 release this summer. We’re working towards that, implementing new features and fixing bugs. As the “code owner” for libc++, I also have to review the contributions of other people to libc++, and evaluate and fix bugs that are reported. That’s a never-ending task; there are new contributions ever day. LWG papers implemented this month. P0811 Add std::midpoint and std::lerp for C++20 LWG issues resolved this month 2960 nonesuch is insufficiently useless 2977 unordered_meow::merge() has incorrect Throws: clause 2164 What are the semantics of vector.emplace(vector.begin(), vector.back())? LLVM features implemented this month (certainly incomplete) Fixed the implementations of list::remove_if and list::unique to deal with values or predicates that are elements in the list. Same for forward_list. We did this for remove already, but now we do it for the other operations as well. Added a bunch of new tests for things that we were missing ** list::sort and forward_list::sort are required to be stable. ** You can’t use match_results until you’ve done a regex search. Our tests did this in several places; now we have assertions to prevent that. ` LLVM bugs resolved this month (probably incomplete) Bug 41323 Race condition in steady_clock::now for _LIBCPP_WIN32API Bug 41130 operator/ of std::chrono::duration and custom type. Bug 41577 test/std/utilities/optional/optional.object/optional.object.ctor/move.fail.cpp has wrong assumption. I spent a fair amount of time on Bug 39696 “Workaround “error: ‘(9.223372036854775807e+18 / 1.0e+9)’ is not a constant expression”; which turned out to be a GCC bug on PowerPC machines. Also, there was a series of general cleanups in the libc++ tests to improve portability and readability. I added a bunch of updates for debug-mode, and there were several places where we assumed that string::compare returned -1/0/1 instead of what was specified, which is \<0/0/\>0. Also, I added tests for std::any_cast and array types. The current status of libc++ can be found here: C++20 status C++17 status C++14 status (Complete) Libc++ open bugs WG21 There were no WG21 meetings in April. However, LWG held three teleconferences this month, reviewing papers in advance of the July meeting. We’ll have more teleconferences in May. I am working on more “cleanup” papers similar to P1458 - Mandating the Standard Library: Clause 16 - Language support library, and my P0805 - Comparing Containers needs an update. The goal of the July meeting is to have a “Committee Draft” (CD) of the proposed C++20 standard that can be sent out for review. Also on my TODO list is to attempt to implement some of the proposals that are coming up for a vote in July (flat_map, text formatting, etc). Boost We released Boost 1.70 on the 12th of April. Once again, I was the release manager, which involved a bunch of “process management”; things like assembling the release candidates, packaging up release notes, deciding which problems that came up would be fixed (and which ones would not), and updating the web site (and so on, and so on). Conferences This was a big travel month. I gave two presentations: At the LLVM European Developer’s conference in Brussels, I gave a 30 minute overview of the changes that were coming to the standard library for C++20. At ACCU in Bristol, England, I gave a talk titled “Navigating the development and evolution of a library” In May, I will be speaking at: CppNow, May 5-10 in Aspen, CO I have submitted a talk for CppCon in September, but I will not hear back about this for a month or two.

Continue Reading
04/04/2019
Damian's March Update

This month I’ve been working on the following projects: Certify Boost.Beast Boost.Build BeastLounge Certify Certify now properly verifies the hostname of a TLS server according to RFC 2818 or TLS-DANE if available. Additionally, initial support for CRLSets has been merged, although it’s still missing integration into the verification code. I’ve also invested a fair bit of time into researching what other open source libraries do to perform certificate status checking. I’ve looked into BoringSSL, mbedTLS, Botan and even the Go standard library. It’s interesting that no library has a default way of performing the status check of a certificate and it’s left up to the user. The Windows implementation of the certificate store in Certify will now properly use the entire chain passed by the peer, which resolves certificate failures in less common cases. Don’t forget to star the repository: https://github.com/djarek/certify! Boost.Beast Most of the work this month involved making Beast compile faster and use less memory by expanding the code that can use split compilation and reducing redundant dependencies in a few places. Boost.Build I’ve worked on implementing 2 improvements that make it less painful to work with b2: support for finding OpenSSL support for sanitizers in gcc and clang Both are currently still in review. BeastLounge The project lacked functioning CI so I implemented one. Since the project was previously only compiled on MSVC, this proved to be quite challenging, because MSVC accepts code that is not valid C++11. I’ve also created a deplyoment docker image, which allows running the application in popular cloud environments, like Heroku. A development version of the app is available at https://beast-lounge.herokuapp.com/.

Continue Reading
04/02/2019
Marshall's March Update

There are four main areas where I spend my time. Libc++, where I am the “code owner” WG21, where I am the chair of the Library Working Group (LWG) Boost Speaking at conferences This month, I spent far more time reviewing other people’s code and preparing talks for conferences than the previous few months. The Boost release process consumed a fair chunk of time as well. Libc++ The big news is: we released LLVM 8 this month! (March 20th). You can get the sources and pre-built binaries from the LLVM download page, or wait for your system vendor to provide you with an update. As the “code owner” for libc++, I also have to review the contributions of other people to libc++, and evaluate and fix bugs that are reported. That’s a never-ending task; there are new contributions ever day. LWG papers implemented this month. P0811 std::midpoint for integral and pointer types. This turned out to be quite involved, and spawned a clang bug report. On the plus side, now I have a topic for a talk for CppCon this fall. Still to do, std::midpoint for floating point types. This is done, but it needs better tests. LWG issues implemented this month I didn’t actually commit any LWG issue fixes this month. I worked with others on several bug fixes that landed, but not under my name. LLVM features implemented this month (certainly incomplete) Add noexcept to operator[] for array and deque Mark vector::operator[] and front/back as noexcept Mark front() and back() as noexcept for array/deque/string/string_view Make to_chars/from_chars work back to C++11. This lets us use them in to_string. LLVM bugs resolved this month (probably incomplete) Bug 35967 <regex> syntax_option_type is not a proper bitmask No bug # Fix a minor bug with std::next and prev not handling negative numbers. No bug # Cleanup of requirements for optional - we no longer allow optional<const in_place_t> Bug 41130 operator/ of std::chrono::duration and custom type. Also, there was a series of general cleanups in the libc++ tests to improve portability and readability. Eric and I (mostly Eric) revamped the debug-mode support, and there will be more activity there in the future. Also, we’re moving towards using more of the ASSERT_XXXX macros for readability, and I revamped about 30 of the tests to use them. Only several thousand to go! The current status of libc++ can be found here: C++20 status C++17 status C++14 status (Complete) Libc++ open bugs WG21 The “winter” WG21 meeting was held in Kona, HI on February 18-24. This was the last meeting for new features for C++20, and as such, it was both contentious and very busy. Between now and the next meeting (July), LWG will be working on reviewing papers and issues to be adopted in July. We have had three teleconferences since Kona, and a fourth is scheduled for mid-April. I am working on more “cleanup” papers similar to P1458 - Mandating the Standard Library: Clause 16 - Language support library, and my P0805 - Comparing Containers needs an update. The goal of the July meeting is to have a “Committee Draft” (CD) of the proposed C++20 standard that can be sent out for review. Boost It’s time for another Boost release (1.70), and I am acting as the release manager again. The release calendar is available (as always) on the Boost website. The cut-off for contributions for the release is 3-April, with a release candidate to follow close behind, and the actual release to happen on the 10th. Once the release is over, I’ll be putting some serious time into Boost.Algorithm; there are a bunch of C++17/20 algorithms that can be added to the library (among other things). Conferences I had submitted talk proposals to three conferences, and all three were accepted. I will be speaking at: LLVM European Developer’s Conference, April 8-9 in Brussels ACCU, April 10-13 in Bristol CppNow, May 5-10 in Aspen, CO

Continue Reading
03/16/2019
Certify X509 Validation

Certify - X509 certificate validation I always knew that validating a certificate chain presented by a peer is not an easy procedure, but my recent work in Certify to port over the procedure from Chromium has only proven that I underestimated the complexity of it. Certificate revocation seems to be a particularly hard issue, with 2 main categories of solutions - offline and online validation. Online validation - OCSP OCSP is a protocol designed to allow checking the revocation status of a certificate by sending a request over a subset of HTTP/1.1. At first glance, it seems it solves the status checking problem on its own. However, OCSP has problems, inherent to online checking. First of all, the validation server might not be currently available - so a lack of response is most definitely not a state in which a chain can be trusted. Secondly, the check may be slow, after all, it requires connecting to a separate service. Additionally, the native Windows API for certificate verification does the status check synchronously, which means potentially blocking a user’s thread that typically services asynchronous operations. There is a feature that alleviates most of these issues, at least from the point of view of a TLS client, OCSP stapling. Sadly, it’s not very widespread and actually few large services support it, due to the fact that it increases bandwidth requirements. Certify will, at some point support both OCSP status checks on the client side and support for OCSP stapling. The problem here is that OCSP requires a fairly functional HTTP client and ASN.1 parsing. A lot of this functionality is already present in OpenSSL, however, integrating it with ASIO and Beast may be tricky. Offline validation - CRLs and Google CRLSets The traditional method of checking the status of a certificate involves looking up revocation lists installed in the OS’s store, or downloaded by the application from the CA. Unfortunately CRLs have issues - an example would be an incident from a few years ago when CloudFlare performed a mass revocation which blew up the size of the CRLs by a few orders of magnitude, resulting in a requirement to download multiple megabytes of data, turning CAs into a major performance bottleneck. Google came up with a different mechanism, called CRLSets, which involves a periodic download of a revocation list which is created by Google’s crawler querying certificate status over OCSP. This verification method is fairly attractive for applications that run on systems that already have Google products, since this database is shared, which is why I’ve chosen to provide an opt-in implementation in Certify. For now, updating the database will be out of scope, because that requires a few utilties that are missing from Boost at this time (XML, JSON and an HTTP Client). Don’t forget to star the repository: https://github.com/djarek/certify!

Continue Reading
03/06/2019
Gold sponsor of C++Now 2019

The Alliance is a Gold sponsor for C++Now 2019. This conference is a gathering of C++ experts and enthusiasts from around the world in beautiful Aspen, Colorado from May 5, 2019 - May 10, 2019.

Continue Reading
03/04/2019
Marshall's March Update

Monthly update (or, what Marshall did in January and February) There are four main areas where I spend my time. Libc++, where I am the “code owner” WG21, where I am the chair of the Library Working Group (LWG) Boost Speaking at conferences Libc++ The LLVM “branch for release” occurred in January, and there was a bit of a rush to get things into the LLVM 8 release. Now that is over, and we’re just watching the test results, seeing if anyone finds any problems with the release. I don’t anticipate any, but you never know. As the “code owner” for libc++, I also have to review the contributions of other people to libc++, and evaluate and fix bugs that are reported. That’s a never-ending task; there are new contributions ever day. After the branch, I started working on new features for the LLVM 9 release (for this summer). More calendaring stuff, new C++20 features, and some C++17 features that haven’t been done yet. LWG papers implemented in Jan/Feb P0355: Extending to Calendars and Time Zones. You may remember this from last month's update; this is a huge paper, and I am landing it in stages. P1024: tuple-like interface to span P1227: Signed ssize() functions P1357: Traits for [Un]bounded Arrays LWG issues implemented in Jan/Feb (certainly incomplete) LWG3101: span’s Container constructors need another constraint LWG3144: span does not have a const_pointer typedef Enabled a memcpy optimization for const vectors that was surprisingly missing LLVM bugs resolved in Jan/Feb (probably incomplete) Bug 28412 std::vector incorrectly requires CopyConstructible, Destructible and other concepts Bug 39183 tuple comparison operators return true for tuples of different sizes Bug 24411 libFuzzer outputs that crash libc++’s regex engine Bug 34330 error: use of undeclared identifier ‘isascii’ while compiling strstream.cpp Bug 38606 no_sanitize(“unsigned-integer-overflow”) annotation for decremented size_type in __hash_table Bug 40533 std::minmax_element is 3 times slower than hand written loop Bug 18584 SD-6 Feature Test Recommendations Bug 40566 Libc++ is not Implicit Integer Truncation Sanitizer clean Bug 21715 128-bit integers printing not supported in stl implementation Bug 38844 __cpp_lib_make_unique not defined in <memory> Bug 40495 is_invokable_v<void> does not compile Bug 40270 std::basic_stringstream is not working with std::byte Bug 39871 std::tuple_size should be a struct Bug 38052 std::fstream still good after closing and updating content Also, there was a series of general cleanups in the libc++ tests to improve portability. The current status of libc++ can be found here: C++20 status C++17 status C++14 status (Complete) Libc++ open bugs WG21 The “winter” WG21 meeting was held in Kona, HI on February 18-24. This was the last meeting for new features for C++20, and as such, it was both contentious and very busy. The Modules TS and the Coroutines TS were both adopted for C++20, along with a slew of language features. Here are some trip reports: Herb Sutter Bryce Adelstein Lelbach Guy Davidson My part in this was (as always) to chair the Library Working Group (LWG), the group responsible for the description of the library features in the standard (~1000 pages). We adopted several new features for C++20: P0339R6 polymorphic_allocator<> as a vocabulary type P0340R3 Making std::underlying_type SFINAE-friendly P0738R2 I Stream, You Stream, We All Stream for istream_iterator P0811R3 Well-behaved interpolation for numbers and pointers P0920R2 Precalculated hash values in lookup P1001R2 Target Vectorization Policies from Parallelism V2 TS to C++20 P1024R3 Usability Enhancements for std::span P1164R1 Make create_directory() Intuitive P1227R2 Signed ssize() functions, unsigned size() functions P1252R2 Ranges Design Cleanup P1357R1 Traits for [Un]bounded Arrays I wrote five substantive papers for the Kona meeting, all were adopted. Five of them were very similar, all about improving the wording in the standard, rather than proposing new features. P1458 Mandating the Standard Library: Clause 16 - Language support library P1459 Mandating the Standard Library: Clause 18 - Diagnostics library P1462 Mandating the Standard Library: Clause 20 - Strings library P1463 Mandating the Standard Library: Clause 21 - Containers library P1464 Mandating the Standard Library: Clause 22 - Iterators library I was also the nominal author of P1457 “C++ Standard Library Issues to be moved in Kona”, but that was just a list of issues whose resolutions we adopted. Between now and the next meeting (July), LWG will be working on reviewing papers and issues to be adopted in July. I’m planning regular teleconferences (in fact, we had the first one on 1-March). The goal of the July meeting is to have a “Committee Draft” (CD) of the proposed C++20 standard that can be sent out for review. Boost It’s been a quiet couple of months for Boost, since we’re between releases, and I have been busy with libc++ and WG21 activities. There have been a few bugs to chase down, and the dealing with change requests for the libraries whose maintainers have “moved on” takes some time. However, it’s time for another Boost release (1.70), and I will be acting as the release manager again. The release calendar is available (as always) on the Boost website. The beta release is schedule for March 13th, and the final release for 10-April. Conferences I had submitted talk proposals to three conferences, and all three were accepted. Hence, I will be speaking at: LLVM European Developer’s Conference, April 8-9 in Brussels ACCU, April 10-13 in Bristol CppNow, May 5-10 in Aspen, CO

Continue Reading
03/01/2019
Adler & Colvin engaged

The Alliance engages Adler & Colvin to complete IRS Form 1023, Application for Recognition of Exemption Under Section 501(c)(3) of the Internal Revenue Code. Completing this form can be a daunting task because of the legal and tax technicalities you’ll need to understand. Adler & Colvin is a group of seasoned attorneys based in San Francisco, deeply committed to serving the legal needs of the nonprofit sector. The firm brings an unrivaled depth of expertise and passion to its representation of tax-exempt organizations and individual philanthropists.

Continue Reading
01/14/2019
Marshall's January Update

Monthly update (or, what Marshall did in December) There are three main areas where I spend my time. Boost Libc++ WG21, where I am the chair of the Library Working Group (LWG) Boost: December was a big month for boost, and much of the first part of the month was taken up with the release process. I was the release manager for the 1.69.0 release, which went live on 12-December. The final release process was fairly straighforward, with only one release candidate being made/tested - as opposed to the beta, which took three. In any case, we had a successful release, and the I (and other boost developers) are now happily working on features/bug fixes for the 1.70 release - which will occur in March. Libc++: After the WG21 meeting in November, there was a bunch of new functionality to be added to libc++. The list of new features (and their status) can be seen on the libc++ website. My major contributions of new features in December were Consistent Container Erasure, char8_t: A type for UTF-8 characters and strings, and Should Span be Regular?, and a big chunk of [Extending to Calendars and Time Zones](https://wg21.link/P0355R7). This is all pointing towards the January 16th “branch for release”, and for the scheduled March release of LLVM 8.0. As the “code owner” for libc++, I also have to review the contributions of other people to libc++, and evaluate and fix bugs that are reported. That’s a never ending task; there are new contributions ever day. WG21 Being between meetings (November -> February) there was not any special WG21 work to be done in December. There’s an ongoing stream of bug reports, discussion, paper reviews that get done between meetings, and there is a series of papers that I need to finish for the pre-Meeting mailing deadline on 21-January. I have 1 1/2 done, and need to do 3-4 more.

Continue Reading
11/13/2018
Wg21 San Diego Trip Report

WG21 San Diego Meeting Last week was the fall 2018 WG21 standard committee meeting. It was held in San Diego, which is my hometown. The fact that I helped organize it (while I was working at Qualcomm) had absolutely no affect on the location, I assure you. ;-) This was the largest WG21 meeting ever, with 180 attendees. The last meeting (in Rapperswil, Switzerland) had about 150 attendees, and that was the largest one until now. There were more than 270 papers in the pre-meeting mailing; meaning that people were spending weeks reading papers to prepare for the meeting. Herb Sutter (the convener) has been telling everyone that new papers received after the San Diego meeting were out of scope for C++20, and apparently people took him at his word. This was my first meeting representing the C++ Alliance (though hardly my first overall). The Alliance was well represented, with Rene, Glen, Vinnie, Jon and myself attending. For information about how WG21 is structured, please see isocpp.org. I spent all of my time in LWG, since that’s the group that I chair, and the one that has the most influence over libc++, the library that I work on. The big news from a library POV was that we voted to merge an updated paper based on the Ranges TS into the draft standard; which means that (barring catastrophe) that it will be part of C++20. This was a huge paper, weighing in at 220+ pages. We spent several days in LWG reviewing this (and a bunch of time at previous meetings as well). We also moved a bunch (around 25) of smaller papers; too many to list here. Detailed trip reports can be found around the web: Herb Sutter Reddit The next WG21 meeting is in Kona, HI February 18-23rd.

Continue Reading
10/24/2018
Initial Work On Certify Complete

Initial work on Certify complete It’s been mentioned in my initial blog post that I’d be working on a TLS certificate store abstraction library, with the intent of submitting it for formal review for Boost, at some point in the (hopefully near) future. The initial setup phase (things that every Software Engineer hates) is more or less complete. CI setup was a bit tricky - getting OpenSSL to run with the boost build system on both Windows and Linux (and in the future MacOS) has provided a lot of “fun” thanks to the inherent weirdness of OpenSSL. The test harness currently consists of two test runners that loads certificates from a database (big name for a folder structure stored in git) that has the certificate chains divided into two groups. Chains that will fail due to various reasons (e.g. self-signed certificates, wrong domain name) and ones that will pass (when using a valid certificate store). I’m still working on checking whether the failure was for the expected reason. All the verification is done offline (i.e. no communication with external servers is performed, only chain verification). At this point it looks like I should consider, whether the current design of the verification code is a good approach. Using the verification callback from OpenSSL and asio::ssl is quite an easy way of integrating the platform-specific certificate store API it causes issues with error propagation (transporting a platform-specific error through OpenSSL) and may be fairly slow, because it requires certificates to be reencdoded into the DER format so that they can be fed into the platform-specific API. An alternative to this approach would be load the entire root certificate store, along with CRLs and OCSP configuration into an OpenSSL context. This is potentially a little bit harder to get right but may offer better performance (no reencoding required when veryfing certificate chains) and eliminates the issues related to error handling. Further investigation, as to which approach is better, is required. Don’t forget to star the repository: https://github.com/djarek/certify!

Continue Reading
09/23/2018
Gold Sponsor Of Cppcon 2018

The Alliance is a Gold sponsor for CppCon 2018. This conference is the annual, week-long face-to-face gathering for the entire C++ community. The conference is organized by the C++ community for the community. Attendees enjoy inspirational talks and a friendly atmosphere designed to help individuals learn from each other, meet interesting people, and generally have a stimulating experience.

Continue Reading
09/03/2018
Damian Jarek Joins As Staff Engineer

Damian Jarek joins the Alliance as Staff Engineer. Previously he worked on a number of embedded networking projects for a few major clients. As a Staff Engineer he’ll be working on an open-source companion library for Boost.Beast and Boost.Asio, which will abstract away the platform-specific details of acessing system proxy settings and performing TLS verification of a peer certificate chain using the operating system’s key store.

Continue Reading
08/13/2018
Marshall Clow joins as a Staff Engineer

Marshall Clow joins the Alliance as a Staff Engineer. Previously, he worked at Qualcomm for many years. Most of his time is spent working on libc++, the C++ standard library implementation for LLVM. He is also a member of the C++ standards committee, currently serving as the chair of LWG, the library working group. Marshall has been contributing to the Boost libraries since 2001, and is the author of the Boost.Algorithm library. Furthermore he maintains several other boost libraries, and moderates some of the boost mailing lists. Finally, Marshall has graciously taken on the role of release manager for several Boost versions.

Continue Reading
08/06/2018
The Law Firm for Non-Profits engaged

The Alliance engages The Law Firm for Non-Profits for legal representation and services. They are passionate about supporting, advocating for and partnering with non-profits and the people behind them. For more than three decades, those looking for assistance with non-profit law throughout the United States and around the world have relied on the attorneys of The Law Firm for Non-Profits for superior legal and business guidance.

Continue Reading
05/30/2018
Member Of The International Committee For Information Technology Standards

The Alliance is member of the International Committee for Information Technology Standards. INCITS is the central U.S. forum dedicated to creating technology standards for the next generation of innovation. INCITS members combine their expertise to create the building blocks for globally transformative technologies. From cloud computing to communications, from transportation to health care technologies, INCITS is the place where innovation begins. Membership in INCITS allows voting in official WG21 meetings.

Continue Reading
05/06/2018
Gold sponsor of C++Now

The Alliance is a Gold sponsor for C++Now 2018. This conference is a gathering of C++ experts and enthusiasts from around the world in beautiful Aspen, Colorado.

Continue Reading
02/22/2018
Corporate Logo Is Adopted

A new corporate logo is adopted from the conclusion of a contest on Designhill:

Continue Reading
01/16/2018
Vinnie Falco Joins The Board

Vinnie Falco joins the Alliance board of directors as president.

Continue Reading
01/16/2018
René Rivera Joins The Board

René Rivera joins the Alliance board of directors as secretary.

Continue Reading
01/16/2018
Jon Kalb Joins The Board

Jon Kalb joins the Alliance board of directors as treasurer.

Continue Reading
01/10/2018
Glen Fernandes Joins The Technical Committee

Glen Joseph Fernandes, a Boost C++ library author, contributor, maintainer, joins as a technical advisor.

Continue Reading
01/09/2018
Technical Committee Established

The board of directors establishes the Technical Committee, whose job is to inform the CEO and board on all technical matters such as code review, resume review, the quality of papers, and other ongoing work.

Continue Reading
01/05/2018
Administrate Cpplang Slack Workspace

The Alliance is now the owner and administrator of the Cpplang Slack Workspace. This workspace is the premiere and most popular community of C++ enthusiasts and professionals from around the globe.

Continue Reading
12/10/2017
Incorp Engaged As Registered Agent

The Alliance engages InCorp Services, Inc. as the registered agent. InCorp provides National Registered Agent services in all 50 states and Washington, D.C.

Continue Reading
12/05/2017
Foundation Group Is Engaged For Registration Service

The Alliance engages Foundation Group, a non-profit formation and compliance services company. Foundation Group delivers a comprehensive 501(c)(3) registration service with a 100% IRS approval rate.

Continue Reading
10/16/2017
Jens Weller Joins As Advisor

Jens Weller, the organizer and founder of Meeting C++, joins the Alliance as an advisor.

Continue Reading
10/01/2017
Louis Tatta joins as CEO

Louis Tatta joins the Alliance in the role of Chief Executive Officer, to oversee and administer the day to day operations of the company and carry out the mission.

Continue Reading
08/17/2017
The C++ Alliance Incorporates In California

The C++ Alliance, Inc. officially incorporates as a California 501(c)(3) non-profit organization. The company is administered entirely as a virtual entity with no physical office.

Continue Reading
07/07/2017
Beast Is Accepted Into Boost

Beast, an HTTP and WebSocket protocol library written in C++11, becomes part of the Boost library collection.

Continue Reading