Wednesday, December 18, 2019

The Economic Reality of Agricultural Robotics and the Return of Widjets, Part 1 of 4

If by progress in agricultural robotics one means that concepts being explored in the engineering lab are maturing into viable commercial products that farmers are willing to make capital investment in, then there really hasn’t been any progress over the last decade or more. When I look at a trade publication today showcasing the latest ag-robot commercial venture, what I see looks exactly the same as what people were doing 10 years ago.

The easy, and least charitable, explanation is that researchers in agricultural robotics are more concerned with making clever robots than they are in addressing the economic realities that farmers are constrained to work within. I personally think the real answer goes much deeper than this. But I would like to start by addressing this first speculation.

Regarding the economic reality farmers face; not once have I seen any article or research group address the fact that in the farming industry, productivity and profit do not necessarily correlate. In fact, often they are found in inverse correlation. If one farmer has a productive crop yield, chances are other farmers in that region have done as well. At harvest time, this gluts the market causing the price per commodity to go down; at which point the farmer actually ends up losing money. It is not unheard for a farmer to let a field go fallow midseason when it becomes apparent that a crop has become a money loser for the year.

And regarding the question of crop rotation, at least in my area (Pajaro Valley), except for the apple orchards and berry fields, I never see the same crop in the same field for more than one growing cycle. For example, for vegetable crops, it’s a given that there will be several different crops in the same field over the course of any growing season. This means that any ag-robot technology that requires permanent field-installed components will also become a nonstarter for the farmer.

For these reasons, farmers are extremely reluctant to make any kind of capital investment in equipment or infrastructure beyond what is minimally necessary. In other words, telling a farmer, as a sales pitch, that using a certain ag-robot technology can raise his/her productivity is not going to be, by itself, a useful marketing strategy.

For a farmer, a robot will be just another piece of equipment like a tractor or harvester; and, whether purchased or leased, machine payments come every month whether the equipment is working in the field or not. Contrast this with the economic reality of hiring a field worker; when the work stops, the field worker can be let go and the farmer’s financial commitment ends. This is a point universally overlooked. Yes, machinery and robotics, on paper, will appear to be far more productive then human labor. But the financial commitment that comes over the lifetime of ownership of a single-crop ag-robot negates any short-term/seasonal financial advantage it has over human labor.

The large corporate farms, whose operations encompass thousands of acres, can make the risky financial choice of purchasing a large sophisticated single-crop harvester or planter.

But for family farms, whose operations are measured in hundreds of acres rather than thousands, investing in such large sophisticated machinery is an out-of-reach financial decision; which is why smaller farming operations are more dependent upon hand labor.

One needs to be careful in reading the last two paragraphs. One has to make the distinction between mechanization and robotics. There are crops like wheat, corn, potatoes, peanuts, sugar beets, and etc., that lend themselves to a high degree of mechanization. From soil preparation to planting through to harvesting, each phase of the farming operation can be taken care of in one pass and at one point in time, with the handling of these particular crops all done mechanically. Contrast this with crops like berries, green beans or zucchini squash, where the harvesting goes on continually over the course of a plant's growing cycle, and which also require careful handling to avoid damage to both the food item and the plant itself.

When looking at those crops that lend themselves to mechanization, one finds that the farming industry has already reduced the need for human labor down to a handful of equipment operators running large integrated computerized machines. At this current state of affairs, robotics and AI offer little-to-no marginal return on investment. The best the robotics community can offer is to replace the remaining equipment operators with AI-based control systems. But the truth is, having a reliable, conscientious and responsible human operator on payroll represents a far greater return on investment to the farmer than spending money on a complicated computer-based AI control system that leaves them at the mercy of some outside, distant, and often unresponsive tech support.

To appreciate how serious this last situation is for farmers, one only needs to look to the recent controversy over the “right to repair” that has affected John Deere tractor owners across this nation.

Wired Magazine, "John Deere Just Swindled Farmers out of Their Right to Repair".
YouTube, "Right to Repair”.
YouTube, "Tractor Hacking: The Farmers Breaking Big Tech’s Repair Monopoly”.

Where robotics can offer a true return on investment to the farmer is with crops that require special handling: picking fruit and berries and harvesting a variety of the vegetable crops. These are the crops for which the cost of labor is the largest single expense the farmer has to face. And with coming minimum-wage laws and other rules respecting work week hours, these are the kinds of expenses that will break a farmer financially. So, the incentive to replace the human fieldworker with an equivalent AI-driven machine could not be more immediate. For example: YouTube, “Harvesting Squash & Eggplant in Fresno County”.

In other words, agricultural robotics will only/finally arrive commercially on the farm when it can offer the farmer a human fieldworker equivalent.

But, here’s the catch, until there is an electro-mechanical equivalent to biological muscle, such a human equivalent robot is an unattainable dream.

Developers develop the ideas they can based on the current technologies available to them. Electromagnetic, hydraulic, and pneumatic are the only robust actuator technologies available to design with. So whatever ag-robot might be designed, its functionality, by necessity, will always be constrained to what these actuator technologies are capable of implementing.

This last observation brings this post back to the speculation as to why the field of agricultural robotics is not progressing. The answer is that, given the existing technologies available, it simply can’t.

Sunday, December 1, 2019

The T2-Tile Project, Publications, Race Conditions, and Breaking Symmetry in Asynchronous Arrays

To get myself back up to speed with the MFM, I went back and reread Dave Ackley’s two papers on the subject: “Pursue Robust Indefinite Scalability”, Ackley, Cannon, 2011, and “A Movable Architecture for Robust Spatial Computing”, Ackley, Cannon, Williams, 2011.

Then I dove further, reading publications cited in these two papers; at least, those that are accessible on the Internet. There are two main design considerations that have come to mind as I have been reading through this literature.

The first is that for very large and indefinitely scalable arrays, power considerations will dominate any design effort and asynchronous arrays are the only ones that satisfy this design constraint. Why this is so is that, neglecting leakage currents, CMOS circuits only consume power when being switched. For large synchronous CMOS logic circuits, the current consumed with the constant clock switching of 100's of MHz to GHz speeds, becomes the dominant power drain. For example, the datasheet for the fifth generation Intel core processor family lists I-ccmax values from 18 to 40 Amps!

Reducing power consumption in CMOS circuitry means clocking individual logic elements only when necessary. How this works in practice is that each individual processor cell in an asynchronous array is given a ring oscillator to act as its own local clock. The crucial feature of a ring oscillator is that it only turns on when an individual processor cell is accessed, then turns off when the cell’s internal programming has run to a stopping point. Therefore, when not in use, the individual processor cells turn off, effectively consuming no power at all. The potential drawback of this scheme is that once a cell has gone to “sleep”, it will only wake up again when accessed from a neighboring cell; that is, only when its ring oscillator gets “rung” again.

The second consideration came to mind as I read through this paper: “Embedding Universal Delay-Insensitive Circuits in Asynchronous Cellular Spaces”, Lee, Adachi, Peper, Morita, 2003.

It struck me as a good example of the kind of disconnect one finds between the theory of array computation versus the reality of hardware design. In this case, the authors considered the situation of a communication race condition; that is, the situation when two neighboring cells initiate communication at the same time. The question, then, of “who goes first?” comes into play with the possibility of a resultant lockup situation occurring. The authors presented a theoretical fix which they called delay-insensitive circuits. Theoretically, this is a fascinating piece of work in that it shows that every synchronous array is equivalent to some asynchronous version of it. But in reality, at the timescales where such communication race conditions would occur, digital electronics starts to behave in an analog fashion, in which case the paper’s straightforward theoretical results will no longer apply.

Race conditions within asynchronous arrays will always be the norm, not the exception. Some way has to be found, so that when communication is initiated between individual cells, they will automatically know who gets to go first and who waits. Just like in any social group, there has to be some kind of dominance hierarchy imposed on the global array.

In terms of what can be accomplished in a practical sense, one has to break the array’s global symmetry – directions have to be defined: up/down, right/left, top/bottom. Then communication is given preferred directions either based on a rule set or by the addition of a “breathing mode”; that is, a phase during which all communication goes in one preferred direction, followed by an alternate period of time when all communication flows in an opposite direction, with information flowing back and forth within the array something like the tides in the ocean.

This is one of the aspects of asynchronous array design that makes simulation qualitatively different than running on bare silicon. In simulation the programming for the virtual machine can be written to arbitrate any race condition that might come up between the individual CA-atoms. But when you forgo the virtual machine and try to run your asynchronous cellular automata on bare-metal processors, then the processor coding itself has to take care of these race conditions. It’s an extra layer to the hardware design that needs to be added to the MFM concept in order to make it work for real.

Saturday, November 30, 2019

Dave Ackley’s T2-Tile Project, First Thoughts

My interest in asynchronous arrays started with my interest as a physicist in cellular automata (CA) and the thermodynamics of computation. That interest then turned itself into a personal challenge to see if I could actually create hardware that would implement these ideas. It was in researching the topic of very large asynchronous arrays of simple processors that I first ran across Dave Ackley’s YouTube channel. That was several years ago, and I’ve been lurking there ever since. I was excited about his work with his Demon Horde Sort simulation. Now that he has started building actual hardware, I’m watching with interest the T2-Tile Project.

Back when I first became interested in arrays of simple processors, I read through a lot of publications on the subject. But at that time, I don’t recall reading any papers that combined the separate subjects of self-assembling and processor arrays into a single research effort.

Papers that dealt with the concept of self-assembly seemed to focus on self-assembly of electrical elements, using techniques of nano-assembly and DNA to implement this process; that is, self-assembling primitive logic elements that could then be self-assembled further into logical computational structures.

While looking at the subject of array processor IC’s and processor arrays, I found that the published research is all over the road-map. But despite the wide range of approaches found in the literature, they all still have one thing in common; that is, the array elements are the computational mechanism of the structure. This is fundamentally different than what Dave Ackley is proposing with the Movable Feast Machine (MFM). In his case, the underlying silicon electronics is not responsible for the computational action of the array structure; rather, it is the interactions of the CA-atoms that are responsible for computation. The underlying electronics is just the structure that the CA-atoms move upon and reside within.

This is the conundrum of the T2-Tile Project: the CA-atoms are running within a virtual machine, which in turn is running upon a classical computational architecture – the very design constraint that Dave is trying to get away from with his concept of Robust-First-Computing. I don’t want this to be taken as criticism; I find Dave’s work fascinating, just not complete. But then again, he is a software person, not a hardware designer. And as a hardware guy myself, I consider the work he’s done to date pretty damn good for someone whose background is not hardware to start with. What I would like to do in the coming months is to pick up where the T2-Tile Project leaves off and carry it further into the design of the underlying hardware – the complementary construction where the MFM will finally be able to become what it wants to be.

What this means, as far as hardware design goes, is that any processor element that forms the array sub-structure underneath the MFM has to be designed, not as computational element in itself, but rather for its ability to support whatever the CA-atoms are programmed to do. In other words, the CA-atoms and the underlying silicon form a symbiotic structure; you can’t design one side of the concept without designing the other side at the same time. It is for this reason I think that this area of research has never been explored; there is no research group out there that has the breadth of expertise to develop both sides of the “robust-first CA-computation” and “self-assembling processor array” combination within a single research effort.

The first step is to outline the fundamental hardware problem, so here it goes…

It is a problem I’ve run into before in hardware design; trying to take a project that ran within a virtual system, then re-cast it as an FPGA based design. It turns out that there are subtle things that virtual machines allow you to do that you can’t reproduce with hardware alone. Something that software people don’t always appreciate, but hardware designers will, is that a virtual machine can step outside of itself and take advantage of computational processes at the hardware level, that the CA running under the virtual machine can’t. The best way to illustrate this is with an example…

The reason DReg or Res can read what’s in a neighboring cell is that the virtual machine, that the MFM is running under, can take advantage of the fact that all of the data of neighboring cells is contained within the same common area in RAM. But this can’t be done within an array where each cell of the array is hardware-independent from its neighbors. In such an arrangement, each cell only knows what’s in its own memory; to find out what resides in a neighboring cell, some form of hardware interrogation must transpire. This kind of interrogation process is not contained in Dave Ackley’s core programming for the CA-atoms.

The virtual MFM machine also keeps a common library of function calls that each cell can access. The only information then that a cell needs to contain is a single 64(?) bit data string. But this feature doesn’t translate from the virtual machine environment to an array of hardware-independent processor elements either. In this case, each CA-atom has to maintain an independent local copy of all of its programming code.

That is, before we can re-create the MFM at the hardware level, a number of the functional attributes that the CA-atoms currently possess get thrown out, while other features will have to be added. That’s where the hardware design becomes challenging. Basically, short of inspired brilliance, one has to work by an iterative process of cut and try; propose a primitive computational structure for the underlying hardware elements, then try out complementary sets of instructions for the CA-atoms, run it all in simulation, and see what happens; then rinse, repeat!

Sunday, July 1, 2018

Emergence

The last of the topics I’ve devoted time to these last months is the subject of emergence; sometimes referred to as emergent behavior/phenomena/systems. It’s an area of inquiry that crosses boundaries from philosophy and theology all the way to physics, chemistry, and biology.

This additional area of interest is the natural continuation of my interests in systems of distributed intelligence and cellular automata as computational systems. That is, both of these areas of inquiry take you right to the heart of emergent behaviors. To jumpstart my research in this area, I’ve started reading the book “Re-emergence of Emergence, The Emergentist Hypothesis from Science to Religion”, edited by Philip Clayton and Paul Davies.

So far this book seems to be a very good introduction to the depth, breadth and history of the cross discipline discussions on what constitutes emergence.

To go along with this added dimension to my interests, I’ve now added the keywords emergent-systems underneath the blog header.

Friday, June 29, 2018

Whither goes K-12 STEM education? Is it time to bring back shop classes?

The second of three topics that occupied my thoughts these last few months was the question, where stands STEM education in K-12?

After spending several days trying to put my thoughts into words, I had to give up. There’s no way to engage in this topic without finding oneself drawn into the politics of K-12 education. And the last thing I want is for politics to show up in my blog. So, I’ve abandoned any attempt at a discussion of the subject.

The only aspect of my thoughts, that I think I can express without getting drawn into a political discussion, is to note that the robotics/technology side of STEM education would be far better served if it were taught within a traditional shop class format. But, the traditional shop class is now inextricably associated with the older practice of tracking; a practice which has become much maligned within today’s educational communities.

So I’ll just put the question out there and move on to other things.

Wednesday, June 27, 2018

Willow Garage’s Legacy, a Blessing or a Decade’s Long Detour in the Evolution of Robotics?

One result of my not being able to work on electronics for the last year was that I had a chance to ponder other questions. One of these questions was wondering about, "whatever happened to Willow Garage?" There was a time about a decade ago it was heralded as one of the great innovators of the robotics community. Then it just sort of disappeared. It had a number of spinoffs; but none of them ever turned into, what you might call an above average commercial success. It seems that the software innovation ROS has been left as Willow Garage’s only remaining legacy to the robotics community.

Despite WG’s universal acclaim, there was always something about that operation that bothered me. I was never able to quite put my finger on it. So with time on my hands, I “googled” Willow Garage to see if I could find any links to posts that might have made critical comments about it; to see if anyone else might have picked up on what I might be sensing.

Nothing!

As critical and contrary-for-the-sake-of-contrariness as some people can be on the Internet, you would’ve thought that there would have been at least a few critical posts or articles to be found.

So what was I seeing in WG that everyone else seems to have missed? I think I finally have an answer that I can articulate. So here it goes.

The formation of Willow Garage brought together some of Silicon Valley’s top-tier talent. Not only was WG’s initial formation generously self-funded, but over time it was able to attract even more venture capital to fund its ambitious creative efforts. The enthusiasm that WG brought to the robotics community attracted a cadre of dedicated and very talented engineers and programmers. You might be forgiven if you started to see WG as a sort of modern-day robotics Camelot.

But here’s the nagging question; if this is the level of funding and talent it takes to do robotics, then how will robotics ever be able to leave the engineering lab and move out to the farm field, the construction site, or the logging or mining operation?

For example, whatever commercial value an agricultural robotic-field-worker might have to a farmer, it must compete with its $25K a year human counterpart. This basic cost of employment puts a severe cost-constraint on any robotics system intended to be used in the field.

The second and more critical issue is that the people who will be selling, operating, servicing and maintaining field-deployed robots in the future, will by necessity be the same people that are doing those jobs now as regards to farm, construction, logging, or mining equipment. In other words, any robotic system deployed in the field, that requires the additional technical support of a team of Stanford University engineering graduate students, is a nonstarter.

To put it in another way, WG’s approach to robotics completely bypassed the questions of cost, manufacturing, operation, service, and maintenance; all absolutely critical elements for any robotic system to be commercially viable in the field.

What a field-deployed robot needs to be is modular. Its mechanical construction needs to be based on interchangeable subassemblies. And its computational architecture should come in the form of pre-programmed bricks or modules connected together using a single shared serial interface to form a system of distributed intelligence.

This form of construction allows for easy manufacture, easy maintenance, and easy service. Programming is not part of this paradigm. If one wants to change some functionality in a robot, just swap in a different module. The upside of this kind of construction is that this is the level of service, maintenance, and rebuild competency that already exists within the workforce currently employed in the industries of farming, construction, logging, and mining.

This last observation returns us to the question of ROS as being a useful addition to the robotics community’s programming toolkit. Sadly, to run ROS is to become dependent on a particular type of supporting hardware architecture; a computational architecture which is the complete antithesis of what needs to be implemented before robots will leave the engineering lab and proceed out to the field.

So this is my pondering, will ROS, rather than the boon to industry it was held out to be, in the years ahead, turn out to be a decade’s long detour in the evolution of field-deployable robotics?

Tuesday, June 26, 2018

Last Year’s Hiatus

A year ago, last June; I was diagnosed with dilated cardiomyopathy. In hindsight, I can see that its onset was probably around March of that year. At that time, I was running 30 to 40 miles a week on the trails at a local state park. But I began to notice my usual 6 or 9 mile runs were getting slower and slower. Then my runs got shorter and shorter. By June, I couldn’t climb to the top of the stairs here at the house without getting out of breath.

It’s not certain what caused this condition. My cardiologist’s best guess is a possible viral infection. I seem to have fallen victim to a condition that usually hits younger and healthier people. Whatever it was that damaged my heart muscle, its onset was most likely around February/March of last year. But being in such good shape to start with, it took several more months for my condition to deteriorate to the point that it was no longer ignorable.

On the downside, there’s nothing that can be done to help except putting me on blood thinners and blood pressure medications. On the upside though, as my cardiologist has told me, “…except for a weakened heart muscle, [I’m] as healthy as a horse.” An angiogram showed my heart arteries wide open and clear and I have no other signs of cardiovascular disease anywhere in my body. I guess a lifetime of endurance-level physical activity had left me, at the age of 65, in exceptionally good health.

So the good news was, I can’t possibly have a heart attack. A defibrillator was implanted last November. With that addition I felt braver and started working out again.

Then, just when I thought I was getting better, I had a TIA (transient ischemic attack), a mini-stroke. I’ve fully recovered, but it left me fatigued again. Hence my further delay getting back to blogging.

I’m one of those people that, in order for my brain to work, my body must be physically engaged as well. Some people call this being a kinesthetic learner. When I was a kid, I was just called fidgety.

Making a long story short, finally getting back to being physical activity again is enabling me to be mentally active again, too.

So time to restart this blog, and finish up where I was last February.