How can you be sure that your League will be in good hands when you go off board? If you are truly a good leader then you should be prepared!
Create a succession plan that proactively deals with all the things you (or the board) are scared might happen when you leave.
Whatever the fear, make sure your succession plan deals with it proactively to ensure the viability of the organization for the long term.
While the main focus of this plan will be succession, the ancillary benefit is that you will be building organizational infrastructure. And that will provide benefit immediately.
As part of your succession plan, train someone now who could replace you, even temporarily, in the event something happens to you. This doesn’t mean you are going anywhere soon. You may not be leaving for several years! But if the whole League relies on you for its survival, and you really are hit by a meteor tomorrow, then what will happen? Find someone you can share your institutional knowledge with, and train them to share the load now, while you still can.