@dennisdang I spend so much time thinking about this sort of thing, It so nice seeing other people think about it too, and I think you are on the correct track For everything I am about to say please keep in mind we are very very very early in all this, so we have so far to go, but hopefully you can see the some of the ground work and foundation being laid now.
We very much took the approach that we should send the things to be signed to the Private Keys instead of trying to get the keys to the thing that needs to be signed.
You are correct in that we are creating a buffer. We encode that buffer as a hex value and send it in a json payload we call a signable to the wallets. Internally to FCL and the JS-SDK the signable is what we pass into the signing function, in the case of FCL we then route that signable to the wallets based on config we received during authentication (Identity as Configuration).
In NBA TopShot, if you go to pay for something using FLOW (Dont think this is currently enabled for external people) It actually works very similarly to what you described, the backend produces everything the transaction needs (it then slots those values into the sdk, but technically this could be skipped), FCL then takes that data, talks to the wallets, gets back signatures, then sends that data back to the backend, where the backend pulls that data apart and submits it to the chain via the GO-SDK. Admittedly this is currently clunky, but over time we should see these align more and more.
We can conceptually take this a bit further too. The following concept will be hard to make work nicely with something like ledger, especially in the beginning, but the way Blocto on FCL and the way Dapper will work with FCL, they create back channels between dapp and the wallet. It is entirely feasible that you can sign a transaction remotely from your phone with out ever seeing an iframe or pop up in the dapp. The architecture and implementation already allows for this. Those backchannels happen via routing that FCL currently gets during authentication, but it is entirely possible that this routing could be discovered in other ways, and wallets that support this (I’ve been calling Remote Asynchronous Signing) could enable an entirely different world of applications to appear.
Imagine for instance you are in a shop, and they are offering a “Pay with Flow” option. They can initiate the transaction from their ipad, and once getting your Flow Address, the application can route the transaction to your phone for approval, your phone gets a push message, you can then approve or decline the transaction from your phone. Approving it signs the transaction using the Private Key that has never left the HSM in your phone.
We are still a ways away from this stuff, but I think we are trending towards it, and I also definitely think the path in that direction starts with the very same questions you are asking here.