The cheapest protein sources are vegan – lentils, tofu, beans and rice. If you think about it, most traditional diets worldwide are predominantly vegetarian or vegan. In most countries, meat is a luxury item. It's only due to factory farming and government subsidies that meat is cheaper than vegetables in the United States.
While some of the specialty products, like coconut ice cream, are expensive, it's easy to be veg on a very low budget, especially if you can cook for yourself. We recommend the websites Plant Based on a Budget and Plant-Based Meal Plan for low-cost recipes.