I’m sensing several issues. First, you don’t sound as if you agree with the advice that the “integrity issue” warrants termination. (If this advice is coming from the Chairman, it does not matter whether you agree or not.)
A true “integrity issue” is a breach of trust with respect to the organization’s essential values. Nothing destabilizes an organization more than a loss of trust for, and among, its leaders! If the behavior warrants termination, then the senior leader should be asked to resign immediately. Long term stellar performance may influence the separation package, but not the question of termination for cause.
If the integrity issue is less than it appears to those advising you and does not warrant termination, you at least have some damage control and rehabilitation issues to handle. Some rebuilding of honesty, openness and trust (“HOT” Relationships) among the team members will be required.
Whether the senior leader leaves or stays is not as important as to why and how you handle this matter. As the CEO, you must be crystal clear as to what is the right way to handle this situation. Your integrity, your authenticity as a leader, is what is being tested. If the remaining manager, and everyone else who is looking to you for leadership, see your behavior as congruent with the organization’s Values, Purpose, Vision and Priorities, then stability will be ensured.
It sounds as if you really need a safe environment in which you can openly examine and reflect on your options and the consequences of each. Good Luck as you feel your way through this very difficult situation.