Benjamin D. Lisle, Colby College