A neat little phrase, in that the two verbs go well together in terms of meaning, but also how they look and sound. But is that just a coincidence?
Well, the most interesting thing to discover first would be if both words are just constructed from for- being placed before the verbs to give and get as a prefix. That’s certainly the case with forgive.
It comes from the Old English forgiefan, which meant give, remit, pardon, give up, or give in marriage. Forgiefan was a combination of the prefix for- (meaning completely) and the verb giefan (to give). The modern sense of to pardon came along in the late Old English period, and came from the idea of giving up the power to punish somebody.
What about forget? It’s a similar case, in that it’s also a case of adding for- as a prefix. It’s just a different for-. It also dates back to Old English, and the main verb in this case was gietan, meaning to grasp or get. For- in this case probably meant away, and forgietan therefore basically meant to un-get something, or lose it from your mind. So yes, it is basically adding for- to get. The interesting thing about to forget is how the get part is so similar to how we use it nowadays. We get a joke or a concept, meaning we understand it, keep it in our mind, the opposite of losing something from our minds when we forget it.
So forgive and forgt is a catchy phrase, but there’s an interesting story behind it too.