Vila doesn't look shabby but crawling around in the tunnels, he wouldn't be wearing his good clothes. Charles doesn't realize this and as Vila noted, he is a snob so he has skewed opinions of people in lower-grades.

Latent memories? Yes partially. It explains the relationships they seem to instantly develop when they're together...but it doesn't necessarily explain how they got together in the first place, if that makes sense.

Avon is referring to Charles when he's making the comment about the principles. He's not pleased with Charles during this conversation.