Processing data in a separate thread in C++ Builder 6

[msvilans]

Hi,

I would like to use a thread separate from the main UI thread in C++ Builder to process the acquired data. This is for a data acquisition program for a helicopter geophysical instrument.

From what I understand (from reading documentation and experimenting with code) all of the data processing events the Armada stream components execute in the main UI (VCL) thread.

In a previous topic, Chris Smith mentions there are "there are ways to get at the data in a background thread before it is split and moved into the main thread context." How is this done? It sounds like exactly what I would like to do.

The motivation behind this is that the program will have a largish volume of data moving through it (about 800KB/s). Because of this, I have had problems where delays in the UI thread can cause buffer overruns and lost data. I would like to completely isolate (in terms of threads) all UI stuff from the data processing stuff. This would permit me to increase the data processing thread priority, in the hope of minimizing the chance of lost data.

For now, I catch the raw block output from a ChicoPlus in a TIIBlockBuffer in the main thread, and immediately copy it and pass it off to a processing thread. I do the channel splitting myself, and I have taken extra care to make all of the UI thread code as lean as possible. This works quite well for the most part, except if the main thread delays for whatever reason, buffer overruns can still occur.

The main UI thread only receives intermittent snippets (at 10Hz) of data for visualization, using a simple but effective Win32 message based mechanism. After the data is processed, it is again passed off to a different thread for disk writing, using Win32 asynchronous i/o.

All of this works quite well, except I still don't like having all of the data pass through the main UI thread before it reaches the data processing thread. Buffer overruns are now rare, which is good because helicopter time is expensive. The CPU usage is around 5% or less on a 2.4GHz Pentium 4.

In summary, to recap, my question is how to get all of the incoming data directly into a separate data processing thread, without touching the UI thread?

Thanks very much,
Markus.

[csmith]

Markus -

Well, in my message I was only half right. You can get the data before it is split, as raw blocks. But the call to handle this block is moved into the main UI. At one time, this was not the case, but eventually problems with the UI being called in the background thread made the change desirable.

From your description, it appears you are doing most of what I initially was going to suggest. You are copying the raw data and sending it to a background thread. If you are doing any visualization, it is throttled down to a subset of data. We have a Throttler mechanism to do that with the internal splitter. But since your data rate is low, even if that process is inefficient, it should not be a major time sink.

We have not found that the process of moving to the main thread context in and of itself takes much time. In captures to memory we have been able to get gap free data to memory buffers at rates of 40-64 MB/sec.
Writing to disk files is slower, but not so slow that .8 MB/sec should be unachievable. I beleive that even with the splitter we were getting at least 5 MB/sec. And this was with slower machines than today's.

From your description, you say that buffer overruns are rare. If the application cannot process blocks quickly enough for the incoming data rate, all runs should fail at some point - and just about the same point. Armada has internal buffer storage that will slowly fill up and then you get the overflow. But since your CPU is at 5% this seems to be unlikely. CPU usage this low implies that you are keeping up.

Armada uses high priority background threads to move busmastered data into a local cache of about 16 buffers. The system can busmaster 8 buffers before exhausing the memory it can fill before the host responds. If the UI stops processing that means there is 20 odd buffers that can be filled up before any overflow can occur. For reasonable buffer sizes, this can be
a lot of time. A UI operation that holds the system off this long could be considered almost pathological. The number of in memory buffers that are allowed can be increased to override this disturbance.

Essentially, the internals of Armada are doing what you want your high priority thread to do - cache blocks to override small upsets. Also, they run at elevated priority themselves.

On the other hand, if the PCI bus is overloaded and busy, there is only a small amount of caching in the ChicoPlus and on the module before data loss occurs. This seems a more likely cause, as the timescales are shorter.
Also this device may not even be associated with the application. Some system process can possibly trigger activity on the bus that holds off the I/O for too long.

To check where the problem lies, you could try removing the GUI calls, and the disk I/O part and see if you get overflows. You could increase buffer sizes and the number of cached buffers in the pool and see if this helps. If the problem is truly not related to the application processing, you may find overflows even in the case when you throw all the data away. Another tack
is to try measuring the time between buffer deliveries and see if there are any large gaps due to other tasks grabbing the bus. You could also measure times to do GUI and disk writes looking for anomolies.

We had a similar problem with a customer where a disk write operation would take 15 seconds to complete once a night or so. Apparently the disk system was busy doing something zany. Windows is not a true real time system, and sometimes it shows it pretty blatantly.

I hope these suggestions help you find the problem. Perhaps with more data on what triggers the overflow we can find a solution.

[msvilans]

Hi Chris,

Thank you for the informative response and for the suggestions for trying to isolate the problem. I have actually tried almost all of the things you suggested.

In general, I am getting the impression that I am as close as one can get to processing the data outside of the main UI thread. That is, copying the raw blocks immediately and passing them off to a background thread.

A number of things have decreased the frequency of buffer overruns:

- Removed all mutex and/or critical section locks in the main and background data processing threads. Some obscure priority inversion condition seemed to be occurring infrequently, causing the threads to lock up long enough to create a buffer overrun.
- Because thread synchronization is still necessary, I converted all background threads to message-loop based state machines. From research I believe the PostThreadMessage, GetMessage and PeekMessage API functions are lock-free.
- Disabled practically all of the Windows XP eye candy (drop shadows, menu animations, etc.). Doing this alone decreased the buffer overrun frequency, even without the changes to the thread synchronization styles.
- Increased the Caliente buffer period to 30 seconds. (Set the TIICaliente BufferPeriodMax and BufferPeriod properties to 30000 milliseconds.) The larger buffer sizes seem to insulate against most Windows time-outs.

After all of this, the buffer overruns appear to be history. Only on a couple of hardware setups do they still occur (but rarely). I still haven't figured out why, but I suspect these could be a cases where Windows itself could be to blame.

In summary, the buffer overruns were eliminated as soon as I better understood the Armada thread structure, and how to control the Armada buffer sizes. Also, this work was a bit of a crash course in debugging thread conflicts. Thread conflicts in non-real-time type applications are usually not pathological, but in data acquisition applications I was forced to understand exactly how the threads will interact with each other in order to avoid any potential lock-ups.

Disk writing at 0.8MB/s is of course trivial for today's hardware as you pointed out. All the same, I have a background thread that uses the WriteFileEx API function with the callback mechanism to asynchronously stream data to disk. It works like a charm, consumes no noticeable CPU power and doesn't block the execution of the program if there is a temporary delay writing to disk. (If your customer still has the problem of 15 second disk writes, asynchronous i/o may be the solution?)

As a final step to remove load from the main UI thread, I am currently developing an OpenGL based visualization component to take advantage of hardware video acceleration. Native Windows GDI calls are slow and suck up a lot of CPU power. In contrast, with hardware acceleration you can draw polylines with 50000 or more points at 10Hz refresh rate with virtually no CPU load.

Are there any plans to release your ChicoPlus SDK products with support for the Windows CE operating system? Reports indicate that WinCE is a hard real-time OS. I would gladly make the change, since most of my Win32 programming experience would carry right over. Also I would welcome the opportunity to develop for a more stable operating environment, where the OS behaves predictably.

Sorry for the long post. I am posting this information here in case other customers of yours experience similar problems, and might need a starting point to find solutions or things to try.

Thanks again for your reply!

Best regards,
Markus.