Stocks drifted lower after the top U.S. trade negotiator pushed back expectations for a deal that addresses the underlying trade tensions with China.