Concurrent Futures Example

grahamspam8177 | 2020-10-28 22:07:49 UTC | #1

Hello, I am developing an app which needs to request data from several URLs and the JSON response from each needs to be shown in different boxes in the UI. Some of the requests take a long time (one takes over 5 minutes on average). In the caveats section you mention using concurrent futures to solve this problem, and I guess some sort of queue in/queue out could be used also. Can you point me to an example that uses a good approach? the concurrent futures approach? Thanks in advance!

---

grahamspam8177 | 2020-11-06 00:28:33 UTC | #2

Can anyone show an example? Since one request takes so long maybe a 'working' graphic of some sort?

---

martin | 2020-11-10 20:22:44 UTC | #3

Hi @grahamspam8177 nice to see you in the forum. Sorry for the delay in getting back to you, I had some work to do on the main site and got sidetracked.

I'd be happy to put together a quick example for you, but I have a few questions about how you want it to work ---

1. You mention putting the data from the responses in different "boxes" is this a table view (i.e. each response getting a line in a table) or are you updating individual fields with data from separate APIs?
2. Re: the queue in/out, what are you wanting to achieve by using the queue? (You might for example want to throttle the requests.) You don't need a queue to update the UI from multiple jobs, we can connect each job to it's own target widget for example.
3. For a working graphic, where should this be displayed? Is it a global "working" state, or do you want to indicate next to individual fields. If it's global do you want all fields to update at the same time.

You can do this using concurrent futures, but I'm not sure it'll actually help much here -- but will know better with the above details. If you're just connecting to multiple API (urls?) and want to wait for all to complete before returning the data, then Python async (probably within a control thread) might be best.

We'll figure it out!

---

grahamspam8177 | 2020-11-10 21:38:36 UTC | #4

Martin,

Wow! Super excited you reached out.

Answers:

1. The app connects to different REST api servers, and their json reponses need to be used differently.

a) In the Step 1 window/tab/box a simple json list response needs to become a pick list of names, with maybe a button to clear all or select all, and the user must pick 1+. This list is really long (could be 100s) but is quick to get.

b) In the Step 2 window/tab/box the json list from a different REST api server, but same idea: the user must pick 1+ or all or reset. This list is fairly short (about 30) and quick to get as well.

c) In Step 3, the selections from the above two lists are sent to a 3rd REST server. Depending on the selections, the response may take just a few seconds, or it could take 5 minutes. The response is a complex json structure that I need to show the user when it finally arrives.

So, it is a whole-sale replacement of each data in its own window/tab/box. It is kind of a workflow: first pick from a, then pick from b, then do c and get results, but the user can go back and forth to change his selections in A and/or B then in C press the button to ask for the report and wait.

1. I am trying to learn how to deal with parallism/shared data between the UI and its working underpinnings. If I were using C++ I might use thread-safe signals to send commands+args to a thread pool, and the workers would send signals back to the UI to receive responses and refresh the screen. I had read that queues might help , but I have little experience in this regard, while needing the UI to be responsive and kept up to date. So queues not a requirement just my naive solution.

2. in each window/tab/box i thought some typical rotating/circular icon that shows the system is working and waiting for responses, with an abort button in the middle. Or whatever is simple.... just to let the user know the app is ok and to give her the ability to stop.

I hope this explains better. And I super appreciate your expertise as I learn!

---

grahamspam8177 | 2020-11-17 18:51:07 UTC | #5

To generalize, I need help with 1. a UI with a tabs look/workflow feel in which each has its own inputs and outputs 2. a control that acquires data that is then converted into a checkbox/pick list 3. an example of a lengthy/long duration timed operation (with feedback while waiting) 4.mixed in with the above how to safely acquire new data, keep the data fresh and usable among the tabs Thanks again

---