You would probably just have to grant them an easement for the fence.
We will have to do something similar for our neighbors garage. It may be sitting 5-10' over the line, depending where the line gets drawn. The people I bought the house from said something to that effect, and now I see why.