Utilize caching to solve ram problem #17

New Issue

2024-11-27T18:51:25Z

harrowmykel commented

2024-11-27 18:51:25 +00:00

(Migrated from github.com)

Any Interestin using caching?

          i have 6gb of vram left, should be able to run a small LLM? but the gpu is basically always at 100% usage due to it constantly churning out new clips, so i don't know if it can handle another llm. i definitely CANNOT afford to spin up another gpu 😭

Originally posted by @kennethnym in https://github.com/kennethnym/infinifi/issues/11#issuecomment-2285008188

Any Interestin using caching? i have 6gb of vram left, should be able to run a small LLM? but the gpu is basically always at 100% usage due to it constantly churning out new clips, so i don't know if it can handle another llm. i definitely CANNOT afford to spin up another gpu 😭 <img width="88" alt="Screenshot 2024-08-12 at 23 26 40" src="https://github.com/user-attachments/assets/17214759-eb6b-4378-aae4-ed4b3ff034d4"> _Originally posted by @kennethnym in https://github.com/kennethnym/infinifi/issues/11#issuecomment-2285008188_

kennethnym commented

2024-11-27 21:16:16 +00:00

(Migrated from github.com)

the llm itself takes up a lot of vram, has nothing to do with it constantly generating audio. it uses that much ram regardless of whether it is doing something or not

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: kennethnym/infinifi#17