The project with my “beloved” Pic18 is nearly over, the one I’m starting is based on a Freescale ARM with enough horse-power to run a modest desktop. Nonetheless an interface (to the proprietary field bus) is still. . . You guessed, a glorious PIC 18. This is one of the reasons I keeping an eye on the Microchip tools strategy for these chips. The other one is that the nearly over project is likely to be maintained in the coming years to implement small changes and requests from the customer. So, what happened in their yard since the last time I wrote?
First, mplab has been driven into retirement and replaced by a customization of Netbeans, named Mplab X. This I consider a good move since I much prefer this IDE over the sadly ubiquitous Eclipse. The reason is long to tell and may be partly invalidated by the latest progress, but that’s enough matter for another post.
Mplab X is still young and that may be the reason for some clumsiness in migration from Mplab 8.
The other strategic move that Microchip is doing is the unification of the two families of compiler. Microchip had the low cost, under-performing mcc18, then re-branded as mplabc18, and the acquired, high cost, high performance HTC.
Microchip decided to dump mplabc18 and to add some compatibility into the HTC so that mplabc18 code could be smoothly recompiled. This new version has been called xc.
Given what I witnessed using mplabc18 for over 3 years, I thought this had to be a good idea.
Then I took a look at the xc8 manual and I discovered that the compiler is not capable of generating re-entrant code. Despite of this Microchip claims that xc is ANSI (89) compliant. I could rant on the fact that c89 is no longer the c standard at least since 12 years ago, but I will concentrate on re-entrant code (or lack of thereof).
A function is re-entrant when it is computed correctly even if it is called one or more times before returning.
The first case is the recursive algorithm. Many algorithms on trees are formulated quite intuitively in a recursive form. Also expression parsing is inherently recursive.
Do you need recursion on a PIC18? May be not, also because – as you may remember – the little devil has a hardware call stack of only 31 entries.
Is there any other case you need re-entrancy? Yes pre-emptive multi-tasking – a task executing a function may be pre-empted and another task be switched in, if you are unlucky enough then it may be entering the same function (in multiprogramming unluckiness is the norm).
You may correctly object that multitasking on an 8 bit is overkilling. I agree, even if I don’t like to give up options before starting to analyze the project requirements.
Anyway there is a form of multitasking you can hardly escape even on 8bits – interrupts.
Interrupts occur every time the hardware feels it right to do so, or – more precisely – asynchronously. And it is perfectly legal and even desirable to have some library code shared between the interrupt and non-interrupt contexts.
When I pointed this, Microchip technical support answered that this case is solved through a technique they call “function duplication” (see section 5.9.5 in the compiler manual). In fact if you don’t use a stack for passing parameters to a function you may use an activation record (i.e. the parameters and the space for return value) in a fixed position of the static ram. Of course this causes your code not to be re-entrant (that’s why you usually want a stack). But you can simply provide a two levels re-entrancy by duplicating the activation record and the right one according to the context you run in.
Note that you don’t duplicate the code, just the activation records.
Neat trick, at least until you realize that a) PIC18 has two interrupts levels, so you would need three copies of activation records and b) this is going to take a lot of space.
In this sense the call stack is quite efficient because it keeps only what is needed in the current execution point. You will not find, in the stack, anything that does not belong to the dynamic chain of function invocations. Instead, if you pre-allocate activation records and local variables you need that space always.
Well this may not be completely true since the compiler can actually perform convoluted static analysis and discover which functions are mutually exclusive. Say that function a() calls first b() and then c() and that b() and c() are never called elsewhere and that their address is never taken. In this case you may reuse the same memory space for activation records and locals first to host b() stuff and the to host c() stuff.
Note that this kind of analysis is very delicate and sometimes not even possible. In fact you may have function pointers, but you may also have assembly code calling at fixed address. So I don’t think this kind of optimization can be very effective, but I would need to run some tests to support my claim.
Let’s get back to my firmwares. The most complex one counts about 1000 functions. Nearly 700 of them require one or more parameters.
Almost every one of them use local variables.
Even if we count 1 byte per function (and I am underestimating) we are talking of 2k on a micro that has 3.5k of ram.
Yes, I have to admit that the static approach is very fast because addressing is fixed. In any point you know the address of the variable at compile time. You don’t have to walk the stack and direct access is fast.
Anyway what I wrote is enough to demonstrate that it is not possible to port trivially a complex code from mplabc18 to xc8.
A last case when you need re-entrancy is when you have several function wrappers that accepts function pointers. If the wrapped code calls another code that uses the same wrapper you are again in the need for re-entrancy.
Ugly Code
Unfortunately, I’m a sort of a purist when it comes to coding. Code that is not properly indented, global and local scopes garbled up, obscure naming, counter-intuitive interfaces… all conjure against my ability to read a source and causes a headache, acid stomach, and buboes. “Unfortunately,” I wrote, meaning that’s most unfortunate for the poor souls that have to work with me to whom I should appear as sort of source code Taliban.
Recently my unholy-code-alarm triggered when a colleague – trying unsuccessfully to compile an application produced by a contractor – asked me for advice.
More and more I delved into the code, more and more my programmer survival instinct screamed. The code was supposedly C++ and the problem was related to a class, that I would call – to save the privacy and dignity (?) of the unknown author – SepticTank. This class interface was defined inside a .cpp and then again in a .h. Many methods were inlined by implementing them in the class interface (and this possibly was part of the problem).
After resolving some differences, the code refused to link because there was a third implementation of the SepticTank destructor in a linked library. I claimed that such code couldn’t possibly work (even after disabling the dreaded precompiled headers – never seen a Visual Studio project working fine with precompiled headers), even if we could manage to get it compiled and linked the mess was so widespread that nothing good could come.
My colleague tried to save the day by removing the implementation of the SepticTank destructor so as to leave the implementation found in the linked library.
In the end, he had to give up because the code was broken beyond repair, and even if it compiled and linked it crashes on launch (not really surprising).
What stroke me most, basically because it caused a slight fit of dizziness, was the sight of the mysterious operator below –
class SepticTank
{
public:
// code removed for your comfort
operator SepticTank* ()
{
return this;
}
// other code removed, again for your comfort
};My brain had some hard moments trying to decode signals from my eyes. Then it figured out that the coder was redefining the implicit conversion to the class pointer so as to use instances and references where pointers were expected… why on the Earth one should want something like that?!?
Implicit conversions if not handled correctly are a sure way to kick yourself on the nose and this is enough a good reason to stay away. But… trying to enter the (criminal) mind that wrote that code, what’s the purpose? Just to avoid the need for the extra ‘&’ ? Or is it a Javism? Maybe it is better to stay out of such minds…
I’d like to introduce…
Say you have slightly more than one hour to talk about C++ to an audience of programmers that range from the self-taught C-only programmer to the yes-I-once-programmed-in-C++-then-switched-to-a-modern-language. What would you focus your presentation on?I started composing slides thinking C++ for C programmers, but it is a huge task and sure the result won’t fit into a single week.
Also I must resist the temptation to teach, since a single hour is not enough to learn anything.
Then I am planning to re-shaping my presentation in form of wow-slides. I mean every slide (no more than 10-15 of them) should show a C++ idiom / technique / mechanism that would cause a wow-reaction in a C programmer or in a C++80 programmer.
Advices are highly appreciated.
Having a Good Time
Once upon a time, there was this Unix thing powering departmental mainframes. When it came to keeping track of system time it sounded like a perfect solution to have a virtual clock ticking once every second and counting seconds since the EPOCH was the way to keep track of the current date and time. Such a clock, stored into a 32bits, had several advantages – it was simple and easy to store and manipulate, it was granted to be monotonically increasing for such a long time that it would have been someone else problem the day that this grant would have been broken. Now switch context, you are working on a micro within an embedded system. The micro has the standard nowadays System On Chip parade of peripherals, including a hardware timer that can be programmed to interrupt your code at any frequency is convenient to you.
Using 1ms as a clock base may seem a little aggressive, but that timespan makes a lot of sense – it is unnoticeable for us sluggish humans but its sort of eons for a speedy electron. In other words, it is a good compromise to base your time engine. You can build on this some timer virtualization so that you may employ multiple timers either periodical or one shot in your application.
So far so good, but there’s a catch. Consider using a 32bit variable to keep the time (as the glorious Unix time_t). Starting from 0 on every power-up (after all, it is not mandatory to keep the full calendar information), it will take one-thousandth of the time it takes Unix to run out of seconds. Is that enough to be someone else problem? No, that’s much closer to your paycheck. It takes some 49 days to roll over if the variable is unsigned.
At the rollover, you lose one important property of your clock – increasing monotonicity. One ms you are at 0xFFFFFFFF and the next one you are at 0x00000000. Too bad.
Every comparison on time in your code is going to fail.
Is there anything that we could do? Let’s say we have a code like this –
if( now > startTime + workingSpan )
{
// timeout
}
It seems perfectly legit, but what happens if startTime approaches the rollover boundary? Let’s say that startTime is 0xFFFFFFF0, workingSpan is 0x10, and now is 0xFFFFFFF8. That means that now is within the working span and we expect the test to succeed. Instead the test will fail, because 0xFFFFFFF0 + 0x10 rolls into 0x00000000 which is NOT greater than now (0xFFFFFFF8).
Is there any way around before going into 64bits?
You may try to check if the operation is going to overflow and act accordingly, e.g.
uint32_t LIMIT = MAXUINT32 - workingSpan;
if( startTime > LIMIT )
{
handle overflow...
}
Or you may check if the operation overflowed –
uint32_t timeLimit = startTime + workingSpan;
if( now < startTime || now > timeLimit )
{
handle overflow
}
You may have to trim a bit the code for corner cases, but I wouldn’t care because there is a simpler way. An interesting property of modulo arithmetic is that
a - b = modulo[n]( a- b )
as long as a – b < n. That means that if we manage to transform the comparison operation from absolute to relative then we can forget about the overflow and roll over problem. So the following code:
if( workingSpan < now - startTime )
{
// timeout
}
is correct from an overflow/rollover point of view, does not requires extra computing time, nor space, and is as readable as the original code.
The ideal job
It is since long that Videogame Studios recruiters send occasional mail proposing jobs (for whose I am mostly unqualified, but it is always nice to be considered). This morning I received an opening in South France with the following benefit list
- Christmas Bonus for all employees in November = about 1500€ for one full working year and on prorata for a shorter period
- Salary increase each year for good performance (as an example in 2009, salaries increased of 7%, in 2010, salaries increased of 12%, in 2011, salaries increased of 8%)
- Profit sharing system: Above a certain threshold, we give back to employees 50% of bonus gained on video game sales
- Gift vouchers at the end of the year = 140 €
- 19 luncheon vouchers per month of 8,80 € each (we cover 60 % = 5,28€ ), you pay only 40%.
- 50 % of your public transport card will be reimbursed.
- Retirement and health care – we take 100% in charge (for you and your family). And you perhaps know that the French state guarantee for a high level of health care insurance too.
- We contribute to an agency that can helps you financially in buying or renting a flat
- We will help you to find a flat to rent – we have a partnership with relocating company based locally (they help the candidate to find a flat, they welcome him at his arrival, help him with stuff like opening a bank account, discovering the city, finding schools for children, finding a good place to practice sport…..).
- If you’re not speaking French, we would offer you French lessons to facilitate your integration
- Media library : you can borrow consoles, videogames, DVD… for free (see with the ITguys
- Free cakes and soft drinks dispenser.
- As we are more than 50 employees, we work as a work council – Its goal is to propose you some discounted products such as theater tickets, trips, cinema…and another of its roles is to be the mediator between employees and employer if needed.
- From our studio, you will be on the beach in 1 hour and you will be on ski slopes in 2,5 hours !
How couldn’t I be tempted? Especially by the 1 hour distance from seaside?!
Java vs. C++, last call
Well, I dragged this for too long. Now I have to pick the language for the next project. My language of choice is C++, but there are some serious motivation for using Java, one is that all of the junior programmers involved has some experience of Java, another is that middle management is biased toward Java. So, I can still chose C++, but I have to put forward strong and sound engineering reasons. Also because this can become a little … political. I mean – should the project be late for “natural” development reason, or whatever other reason, I don’t want to be blamed because I decided for a language perceived as complicated and rough.
The application isn’t going to need machoFlops or be resource intensive, therefore performance-wise there is no reason to avoid Java.
Java has a thorough and complete library, quite homogeneous even if not always well designed.
I consider C++ to be more flexible and expressive, but is there a more demonstrable software engineering property of C++ that I can use in my argumentations to favor the choice of this language?
(Or conversely is there any demonstrable advantage of Java the could convince me to switch my language of choice).
C++ lambda functions, my first time
Some days ago I wrote my first lambda expression in C++.
int n=0;
std::for_each( begin, end, [[&n]](std::uint8_t& a){a=n++;});
If you are back at the C++85 age, this could look like any other alien language from a Star Trek episode. If you are updated at the more recent C++98, the syntax would look funny at minimum.
At least, that is what it looked to me before starting to get some information about the new standard (you can find a brief yet comprehensive recap of what changed in C++ on this specific page of Bjarne’s website).
You should read the code above as follows. The square brackets defines the intro for the lambda function. Within the square brackets you should put what the lambda function should capture of the environment.
In the code, I stated that n should be captured by reference (&). I could have replaced the ampersand if I wanted to have the capture happen by value. Or I could have put nothing should I wanted the lambda function to capture everything.
Next the argument comes, not different from standard function. Eventually the function body.
Once you get this quick start, you will easily decode as a way to fill a range with increasing integers. Take a breath, get another look at the code, ok, now it should make sense.
Well, I’m quite with Stroustrup when he says that he has mixed feelings about lambda functions (I am looking for the quote, but I don’t have it handy). For simple function lambdas are a godsend. On the other hand, lambdas can yield a terrific potential of hiding mechanism and causing major headaches should they escape your control.
If you compare the line above with the code you have to write previously, it is obvious that lambda expressions are a giant leap forward.
In the C++98 days you ought to write –
class DisposableNameForClass
{
public:
DisposableNameForClass() : n( 0 ) {}
void operator() ( int& a ) { a = n++; }
private:
int n;
};
//..
DisposableNameForClass disposableNameForInstance;
std::foreach( begin, end, disposableNameForInstance );
And that is very uncomfortable and inconvenient. By looking at code like this it easy to question whether it makes sense to use the std::for_each at all rather than roll your own code.
But, look at the traditional code
int n=0;
while( begin != end )
{
*begin++ = n++;
}
This code is pretty clear to anyone has a minimum familiarity with C or one of the derived language (yes, there is the dereference operator which involves pointers, but shouldn’t pose real obstacles to comprehension).
Is it error prone? Possibly as any code longer than 0 bytes. std::for_each saves you at least two errors – messing up the loop termination condition (e.g. forgetting the ! from the comparison, or comparing for equality rather than inequality) and missing the increment of the iterator (this happened to me at least once).
These reasons may not be enough to recommend std::for_each in C++98, but it is hard to argue against if you can use the lambda functions.
A Matter of… Time
As promised, here I am to write about the other bug. This one needs a bit a background to be understood. At the beginning of the current project (that means some 2 years ago) I designed a software timer for providing time services. User code that needs notification about time event may register the software timer specifying the time event. When the time event occurs the user code is called back.
Software timers are identified by a numeric id. Being an 8bit architecture with a severely constrained data memory, I picked an unsigned 8 bit number as timer id. This id indexes a private array containing the software timer parameters.
There are two kind of timers – repeating and one shot. A repeating timer repeatedly calls you back at given intervals, while an one-shot timer just calls you back once when it expires.
Repeating timers need to be explicitly canceled, but when I was designing the class, it felt like a good idea to let one-shot timer self-cancel pending their expiration.
Time passed, Software timer component being tried and tested became trusted. Then I got some mysterious bugs – code that used to work stopped being called back by timer. It took a while to discover what happened. After the one-shot timer executed its callback the user code still had some dangling references to the timer id.
Occasionally user code could inadvertently cancel the timer pointed to by the dangling reference resulting either in an assertion about a non-existing timer (turned into a reset in release code), or in a deletion of an innocent bystander timer.
I thought a while on the matter and I came to the conclusion that it is not good for the timer to self-delete. Rather is must be up to the user code to get rid of the timer. Unfortunately I cannot apply the lesson learned on the current project because the code base is too large and we are releasing, nonetheless I’ll keep a reminder for the next project.
While talking about timer callback, a colleague found another problem – if you perform timer registration and removal during the callback you are in for serious trouble. In fact the code for the timer dispatcher was about like this –
for( id = 0; id < TIMER_COUNT; ++id )
{
if( timerHasExpired(id) )
{
callbackUserCode( id );
if( timerIsOneShot( id ))
{
deleteTimer( id );
}
else
{
// timer is repeat
reloadTimer( id );
}
}
}
This code appears good as long as nothing happens on the timers repository during callback. Consider what happens if a one-shot timer is canceled and a new timer is created in the callback code…
There are two ways to deal with this – either queue timer creation/deletion operation during the callback and execute them when the above code is completed. The other is to re-consider the current time nature after the callback, i.e. consider the timer repository “volatile” across the callback. This is our choice –
for( id = 0; id < TIMER_COUNT; ++id )
{
if( timerHasExpired(id) )
{
callbackUserCode( id );
if( timerHasExpired( id ))
{
if( timerIsOneShot( id ))
{
deleteTimer( id );
}
else
{
// timer is repeat
reloadTimer( id );
}
}
}
}
Lessons learned – 1) don’t provide classes that mixes automatic and explicit object disposal – either the user must always delete objects or the user must never delete objects. 2) critical runs are not just about interrupts, but they may occur in callback as well. Calling back code should be programmed defensively expecting that the user code may wreak havoc on the structure. Should this not be possible then a state variable declaring whether the class is in “callback execution” or “normal execution” must be provided and functions that manipulates class state have to be made aware of the state so to act directly or to defer actions at the end of “callback execution” state.
Ehy dude, where did my frame go?
It is nearly three decades since I write bugs code and I am maturing the idea that each new bug I find should inspire a way to avoid it in the future or, at least, to reduce the chance of falling again in the same mistake. During the past weeks my team and I discovered two bugs and here is my ideas for improving the software quality.
The first bug is likely to be the longest-standing bug I had. As you may know, I am working on a proprietary field bus. We found evidence of the problem at least three years ago. Occasionally our devices lose a frame. It is not tragic – it is a frame in 250 – but, since the protocol does not account for transmission acknowledgment, it may result in wrong device behavior.
We got the assembly driver from our customer, therefore the code is granted to be correct. They use the same code in hundreds of devices and they said they never have such a bad figure in frame reception.
This bug had a chance to live so long for two reasons – first, the customer accepted our firmware with this known problem; second we were quite convinced that this problem should have appeared in their devices as well.
An event broke this impasse – first, we had big troubles in an installation and eventually, it turned out that one device has a slightly defective crystal that caused major frame dropout, wreaking havoc on all the system. This led to some investigation by the hardware guys that told me that crystal precision was very critical in device correctness. The crystal component has a tolerance of 0.5% and our device starts behaving weirdly when the frequency goes 0.4% away from the nominal frequency. This was clearly not acceptable and a solution was urged.
I went to some fruitless investigations before discovering that another firmware, mainly developed by our customer, was much more reliable than ours. After much head-scraping and head-banging against the nearest wall, I went through the aging documentation that the customer gave us to integrate the bus driver.
There I found that it was required to set a global variable, let’s call it FLEX with half of the value of a global constant.
Then I turned to the code and found an old comment of mine that went like this /* FLEX and GIZMO seem not to be used anywhere in the code, they must be private variables */
Of course, removing the comment and properly initializing the variable sent the problem the way of Dodos.
How this could have been prevented? That’s a simple shot – avoid the programmer manually setting obscure and arcane magic values in oddly named global variables. Either provide a complete setup function or let the magic value be computed at compile time.
I’ll write about the other bug, tomorrow. In the meantime, your comments are welcome, as always.
Is this the time for Java?
Once upon a time I was a strenuous ZX Spectrum fan, then time changed, technology evolved and, fighting back my feelings, I switched to Commodore Amiga (yes, it took me a detour over the Amstrad cpc, to get ready for change).Then I was fond of the Amiga, but time changed again, Commodore filed bankruptcy and I changed to anonymous PC clone with Windows 3.1.
Sad story, time changes and you must adapt and change as well, saying goodbye to your comfortable little home.
Now I am faced with a hard decision and it seems that it is time for change again. For the incoming project at the company I work for, I will have to lead the development team (and likely to put in some project management as well, but this is another story) and I have been already told that I don’t have to expect we are going to hire top programmer. On the contrary the company is like to get some instances of Joe Slightly Below Average junior programmer.
The project itself is ambitious (as all projects we tackle), there will be Linux, IP communication, complex system, audio and video communication, proprietary and open field bus communication, zig bee localization and the like… So for sure the team will have to find as much as possible from COTS (Components Off The Shelf) and integrate them.
Now I have to select (apparently I am still free of doing this) the language for the main application. Would have been it my project I would have chosen C++ blindly – no discussion. I am also curious about the new C++11 and this would have been a perfect occasion for getting used to the new tools in the language.
But… I have to deal with junior, possibly below the average programmers (and to make things worse, the development team will be far away from me, but this is another story as well).
From my last weekly report:
I am still undecided about the language to use for this project. There are really only two (maybe three) alternatives.
- Java. Java is quite accessible. It is likely that every college programmer has been exposed (at various degree) to Java. Java prevents you to do simple mistakes on pointers and memory management at the cost of allowing you to enter terrible pains and sufferance by mindlessly making complex mistakes. Java is mature and stable, performance could be sometimes troublesome. Integrating third part components is easy only if they are written in Java or have a Java wrapper.
- C++. C++ is unfortunately a less obvious choice, many people have it on their resume, just to pair C, but are quite lost at what is C++ today. C++ helps you strongly in avoiding the terrible pains and sufferances of complex mistakes, by letting yourself tie a knot around your neck with simple errors of pointers and memory management. Although C++ is with us since a long time, the latest standard is still young and many programmers still don’t have a clue about the previous one. Performance is not an issue. Integrating third party components should be easy.
- Python. Ok, I am not really a script language fan, but I reckon that for integration work, these could be the right tools. Write few “glue” code that just works and ignore the rest. Performance are not what you want to look at, but it is up to third party components to be quick. Though resources can be limited on an embedded system to use a scripting language as it was thought to be used.
Despite the silly wording, Java seems to be the reasonable answer, and for sure is the one preferred by management, but is it the right one?