I would structure it this way.
Tell the builder you only work for the client. Period.
You will however make them look very good for recommending you and will gain them more business and recommendations.
Failing that, walk away.
Builders will either drag you down to their level, or cause you massive problems.
