driveline angles are always (or should be) a concern. however, more importantly than drivetrain geometry is steering geometry. it effects handling, tire wear, and can effect fuel economy.
once you get the steering correct, then worry about driveline. you can grind the CV for clearance. the transfer case drop will actually make your front angles worse.