The caterpillars were all over the fennel last summer, too, but I didn't realize exactly what they were - I just assumed that they were harmful - after all, they were eating my plant. So I picked them all off, dropped them in a bucket, and drowned them. A few days later, the Stokes Butterfly Book I had ordered arrived - and I soon discovered that those caterpillars would have become beautiful Swallowtail butterflies like those pictured below if I'd left them alone.
The butterflies seemed a little late arriving this year, but just in the last week I've seen several beauties. And they're fairly cooperative when I want to take pictures of them.