Yes the quality of the product does matter as does the sales letter and promotional tools the affiliates can use. But you are talking about using your own affiliate tracking and paying the affiliates yourself. If affiliates have never worked with you, how are you going to guarantee you will pay your affiliates when you promise.
When affiliates use vender/affiliate platforms like ClickBank they know without a doubt that they will receive their promised commissions when promised because it is not the vendor that is paying the affiliate but it is the vendor/affiliate platform that will be paying the affiliates just like clock work.