I’ve been passively watching the FPGA space for the past few years.  Partially because I think they’re a really interesting technology, but also because, as The Next Platform says:

[T]here are clear signs that the FPGA is set to become a compelling acceleration story over the next few years.

From the relatively recent Intel acquisition of Altera by chip giant Intel, to less talked-about advancements on the programming front (OpenCL progress, advancements in both hardware and software from FPGA competitor to Intel/Altera, Xilinx) and of course, consistent competition for the compute acceleration market from GPUs, which dominate the coprocessor market for now

I’m not sure it’s as sure a thing as they are making it out to be, but I think there are several reasons to think FPGAs have a good chance of becoming much more mainstream over the next five years.  I think there are some underlying technological forces underway (FPGA’s power-efficiency becomes more and more attractive over time), as well as some “the time is ripe” elements such as the Intel/Altera aquisition and the possibility that deep learning will continue to drive demand in computational accelerators.

One of the commonly-cited drawbacks of FPGAs [citation needed] is their difficulty of use.  I’ve thought about this a little bit in the context of discrete FPGAs, but with the introduction of CPU+FPGA hybrids, I think the game has changed pretty considerably and there are a lot of really interesting opportunities to come up with new programming models and systems.

There are some exciting Xeon+FPGA parts coming out later this year (I’ve seen rumors that Google have already had their hands on similar parts), but there are already options out on the market: the Xilinx Zynq.

Zynq

I’m not going to go into too much detail about what the Zynq is, but basically it is a CPU+FPGA combo.  Unlike the upcoming Intel parts, which look like separate dies in a single chip, the Zynq I believe is a single die where the CPU and FPGA are tightly connected.  Another difference is that rather than a 15-core Xeon, the Zynq comes with a dual-Cortex-A9 (aka a smartphone processor from a few years ago).  I pledged for a snickerdoodle, but I got impatient and bought a Zybo.  There’s a lot that could be said about the hardware, but my focus was on the state of the software so I’m just going to skip to that.

I’ve ranted blogged about how much I dislike the Xilinx tools in the past, but all my experience has been with ISE, the previous-generation version of their software.  Their new line of chips (which includes the Zynq) work with their new software suite, Vivado, which is supposed to be much better.  I was also curious about the state of FPGA+CPU programming models, and Xilinx’s marketing is always talking about how Vivado has such a great workflow and is so great for “designer productivity”, yadda yadda.  So I wanted to try it out and see what the current “state of the art” is, especially since I have some vague ideas about what a better workflow could look like.  Here are my initial impressions.

Vivado

Fair warning — rant follows.

My experience with Vivado was pretty rough.  It took me the entire day to get to the point that I had some LEDs blinking, and then shortly thereafter my project settings got bricked and I have no idea how to make it run again.  This is even when running through a Xilinx-sponsored tutorial that is specifically for the Zybo board that I bought.

The first issue is the sheer complexity of the design process.  I think the most optimistic way to view this is that they are optimizing for large projects, so the complexity scales very nicely as your project grows, at the expense of high initial complexity.  But still, I had to work with four or five separate tools just to get my LED-blinky project working.  The integration points between the tools are very… haphazard.  Some tools will auto-detect changes made by others.  Some will detect when another tool is closed, and only then look for any changes that it made.  Some tools will only check for changes at startup, so for instance to load certain kinds of changes into the software-design tool, you simply have to quit that tool and let the hardware tool push new settings to it.  Here’s the process for changing any of the FPGA code:
– Open up the Block Diagram, right click on the relevant block and select “Edit in IP Packager”
– In the new window that pops up, make the changes you want
– In that new window, navigate tabs and then sub-tabs and select Repackage IP.  It offers to let you keep the window open.  Do not get tricked by this, you have to close it.
– In the original Vivado window, nothing will change.  So go to the IP Status sub-window, hit Refresh.  Then select the module you just changed, and click Upgrade.
– Click “Generate Bitstream”.  Wait 5 minutes.
– Go to “File->Export->Export Hardware”.  Make sure “include bitstream” is checked.
– Open up the Eclipse-based “SDK” tool.
– Click “Program FPGA”.
– Hopefully it works or else you have to do this again!

Another issue is the “magic” of the integrations.  Some of that is actually nice at “just works”.  Some of it is not so nice.  For example, I have no idea how I would have made the LEDs blink without example code, because I don’t know how I would have known that the LEDs were memory-mapped to address XPAR_LED_CONTROLLER_0_S00_AXI_BASEADDR.  But actually for me, I had made a mistake and re-did something, so the address was actually XPAR_LED_CONTROLLER_1_S00_AXI_BASEADDR.  An easy enough change if you know to make it, but with no idea where that name comes from, and nothing more than a “XPAR_LED_CONTROLLER_0_S00_AXI_BASEADDR is not defined” error message, it took quite a while to figure out what was wrong.

What’s even worse though, was that due to a bug (which must have crept in after the tutorial was written), Vivado passed off the wrong value for XPAR_LED_CONTROLLER_1_S00_AXI_BASEADDR.  It’s not clear why — this seems like a very basic thing to get right and would be easily spotable.  But regardless of why, it passed off the wrong variable.  It’s worth checking out the Xilinx forum thread about the issue, since it’s representative of what dealing with Xilinx software is like: you find a forum thread with many other people complaining about the same problem.  Some users step in to try to help but the guidance is for a different kind of issue.  Then someone gives a link to a workaround, but the link is broken.  After figuring out the right link, it takes me to a support page that offers a shell script to fix the issue.  I download and run the shell script.  First it complains because it mis-parses the command line flags.  I figure out how to work around that, and it says that everything got fixed.  But Vivado didn’t pick up the changes so it still builds the broken version.  I try running the tool again.  Then Vivado happily reports that my project settings are broken and the code is no longer findable.  This was the point that I gave up for the day.

Certain issues I had with ISE are still present with Vivado.  The first thing one notices is the long compile times.  Even though it is hard to imagine a simpler project than the one I was playing with, it still takes several minutes to recompile any changes made to the FPGA code.  Another gripe I have is that certain should-be-easy-to-check settings are not checked until very late in this process.  Simple things like “hey you didn’t say what FPGA pin this should go to”.  That may sound easy enough to catch, but in practice I had a lot of trouble getting this to work.  I guess that “external ports” are very different things from “external interfaces”, and you specify their pin connections in entirely different ways.  It took me quite a few trial-and-error cycles to figure out what the software was expecting, each of which took minutes of downtime.  But really, this could easily be validated much earlier in the process.  There even is a “Validate Design” step that you can run, but I have no idea what it actually checks because it seems to always pass despite any number of errors that will happen later.

There’s still a lot of cruft in Vivado, though they have put a much nicer layer of polish on top of it.  Simple things still take very long to happen, presumably because they still use their wrapper-upon-wrapper architecture.  But at least now that doesn’t block the GUI (as much), and instead just gives you a nice “Running…” progress bar.  Vivado still has a very odd aversion to filenames with spaces in them.  I was kind enough to put my project in a directory without any spaces, but things got rough when Vivado tried to create a temporary file, which ended up in “C:\Users\Kevin Modzelewski\” which it couldn’t handle.  At some point it also tried to create a “.metadata” folder, which apparently is an invalid filename in Windows.

These are just the things I can remember being frustrated about.  Xilinx sent me a survey asking if there is anything I would like to see changed in Vivado.  Unfortunately I think the answer is that there is a general lack of focus on user-experience and overall quality.  It seems like an afterthought to a company whose priority is the hardware and not the software you use to program it.  It’s hard to explain, but Xilinx software still feels like a team did the bare-minimum to meet a requirements doc, where “quality beyond bare minimum” is not seen as valuable.  Personally I don’t think this is the fault of the Vivado team, but probably of Xilinx as a company where they view the hardware as what they sell and the software as something they just have to deal with.

end rant.  for now

Programming model

Ok now on to the fun stuff — the programming model.  I’m not really sure what to call this, since I think saying “programming model” already incorporates the idea of doing programming, whereas there are a lot of potential ways to engineer a system that don’t require something that would be called programming.

In fact, I think Xilinx (or maybe the FPGA community which Xilinx is catering to) does not see designing FPGAs as programming.  I think fundamentally, they see it as hardware, which is designed, rather than as software, which is programmed.  I’m still trying to put my finger on exactly what I mean by that — after all couldn’t those just be different words for the same thing?  There are just a large number of places where this assumption is baked in.  Such as: the FPGA design is hardware, and the process software lives on top, and there is a fundamental separation between the two.  Or: FPGAs are tools to build custom pieces of hardware.  Even all the terminology comes from the process of building hardware: the interface between the hardware and the software is called an SDK (which is confusingly, also the name of the tool which you use to create the software in Vivado).  The software also makes use of a BSP, which stands for Board Support Package, but in this case describes the FPGA configuration.  The model is that the software runs on a “virtual board” that is implemented inside the FPGA.  I guess in context this makes sense, and to teams that are used to working this way, it probably feels natural.

But I think the excitement for FPGAs is for using them as software accelerators, where this “FPGAs are hardware” model is quite hard to deal with.  Once I get the software working again, my plan is to create a programming system where you only create a single piece of software, and some of it runs on the CPU and some runs on the FPGA.

It’s exciting for me because I think there is a big opportunity here.  Both in terms of the existence of demand, but also in the complete lack of supply — I think Xilinx is totally dropping the ball here.  Their design model has very little room for many kinds of abstractions that would make this process much easier.  You currently have to design everything in terms of “how”, and then hope that the “what” happens to work out.  Even their efforts to make programming easier — which seems to mostly consist of HLS, or compiling specialized C code as part of the process — is within a model that I think is already inherently restrictive and unproductive.

But that’s enough of bashing Xilinx.  Next time I have time to work on this, I’m going to implement one of my ideas on how to actually build a cohesive system out of this.  Unfortunately that will probably take me a while since I will have to build it on top of the mess that is Vivado.  But anyway, look for that in my next blog post on the topic.