Both “so to speak” and “so to say” (meaning “as one might say”) can be found in English literature and English dictionaries. However, take a look at the following graph, which shows how common the two phrases have been in the history of English literature:
Note: The graph was produced with Google Ngram Viewer by searching for “so to say.” and “so to speak.”, with a full stop at the end to avoid instances such as “So to say that we ‘know’ something is often not a simple matter.”, which are unrelated to the phrase.
As you can see, “so to speak” is about 30 times more common than “so to say”, and the popularity of “so to say” has been slowly but steadily declining since about 1915. Many native speakers claim not to be familiar with the phrase “so to say” at all (which is, in fact, unlikely, but when they hear another native speaker say the phrase, they write it off as regional usage or a slip of the tongue).
The point is that “so to say” is so uncommon that it would be perceived as an error by a large number of native speakers if used by a non-native speaker. My advice to non-native speakers therefore is: Avoid “so to say” altogether and only use “so to speak”.
This may require consciously thinking about the phrase whenever you say it, as the corresponding phrase in your mother tongue is likely to employ a verb usually translated as “say”, as in German sozusagen or French pour ainsi dire, where sagen and dire are translated as “say” when used in isolation.