Others have written good general information about how to attract and retain contributors for your open source project. Here, I'm going to focus rather narrowly on one critical factor in growing your contributor base: speed. Since it's what I'm familiar with, I'm going to write with the maintainer in mind.
The moment a new contributor posts a PR or a draft patch, a timer starts. The longer maintainers take to respond to their submission, the lower the chance that person will ever contribute to the project again. While no one has done a specific study on this that I know of, my experience is that it drops by half in a couple of days and it gets very close to zero in only a few weeks. If you're running a young project and looking to attract lots of new contributors, there may be no higher priority than dealing with first-time contributions quickly. Minutes to hours is the target here.