My role

Environment transition, software installation, CUDA configuration, model testing, prompt experimentation, output evaluation, architectural research, and hardware-reuse planning

Available hardware

Six NVIDIA RTX 3090 GPUs with 144 GB of aggregate physical VRAM

Core environment

GPT4All, CUDA acceleration, a first-generation Llama-family model, and locally stored documents

Experiments

Local chat inference, document-assisted interaction, model loading, multi-GPU testing, hallucination review, and early agent-style workflows

Primary challenges

PCIe communication bottlenecks, distributed memory, software support, energy use, cooling, and output reliability

Final outcome

Confirmed that the hardware could support local AI experimentation, but determined that the riser-based six-GPU architecture was not practical as a unified inference platform

The objective was to determine whether the existing mining platform could serve as a local AI environment without requiring an entirely new system.

The experiment focused on several questions:

• Could the six GPUs be recognized and used within the local AI environment?

• Could models be distributed across multiple GPUs?

• Would the combined GPU memory make larger workloads practical?

• Could the platform support local chat, document-assisted interaction, and experimental agent workflows?

• Would operating performance justify the power, cooling, and maintenance requirements?

I removed the mining environment and prepared the platform for local AI experimentation.

The reconfigured environment included:

• Six NVIDIA RTX 3090 GPUs

• 144 GB of aggregate physical VRAM

• GPT4All

• CUDA acceleration

• A first-generation Llama-family model

• Updated NVIDIA drivers

• Local model files

• Locally stored documents for assisted interaction

• Monitoring and system-management tools

I began with chat-style prompts and later experimented with loading documents so the model could respond using information provided within a local working context.

I managed the experiment from hardware repurposing through evaluation, including:

• Removing the mining environment

• Preparing the system for local AI experimentation

• Installing and configuring AI-related software

• Testing GPT4All and local model workflows

• Exploring CUDA acceleration

• Loading and testing local models

• Testing chat-style prompts

• Loading documents for document-assisted interaction

• Evaluating model responses for usefulness and accuracy

• Identifying hallucinations and unreliable output

• Investigating whether workloads could be distributed across multiple GPUs

• Researching hardware options for better AI performance

• Comparing mining infrastructure requirements against AI infrastructure requirements

• Deciding whether continued investment in the six-GPU platform was worthwhile

The most important discovery was that GPU count alone did not determine whether the platform was suitable for local AI.

The mining platform was designed for workloads where each GPU could operate mostly independently. That worked well for mining, because each card could contribute compute capacity without needing constant high-speed communication with the other GPUs.

Local language-model inference behaved differently. When model layers or data needed to move between GPUs, the system depended more heavily on the motherboard, PCIe pathways, and interconnect bandwidth.

Because the GPUs were connected through PCIe risers, the system could not simply behave like one large 144 GB memory pool.

Design decision: I stopped treating the six GPUs as one combined AI resource and began evaluating whether the architecture matched the workload. The issue was not just how much GPU memory existed, but how the GPUs communicated with the rest of the system. 

The project also revealed a software-side limitation: local model responses could sound confident while still being incomplete, inaccurate, or misleading.

This made output evaluation an important part of the experiment.

Issues Observed

• Responses could appear convincing even when they were wrong

• Document-assisted answers still required human review

• Model behavior depended heavily on prompt structure

• Smaller local models had practical limits compared with larger hosted systems

• Useful output required validation, not blind trust

Operational lesson: Local AI was not just a hardware problem. The system also required careful prompt testing, output review, and an understanding of when the model should not be trusted.

After identifying the platform's limitations, I researched several possible upgrade paths.

Options considered included:

• Replacing mining risers with higher-bandwidth ribbon-style connections

• Eliminating risers and changing the system layout

• Investigating direct GPU interconnect options such as NVLink

• Replacing the motherboard and CPU platform

• Building a system designed specifically for AI workloads

• Moving toward smaller purpose-built AI workstations

These options were investigated as potential architectural improvements, but I did not continue converting the full mining platform into a dedicated AI system.

The cost, power use, cooling requirements, and hardware changes required to make the platform better suited for AI did not justify the likely benefit.

The experiment involved repeated testing and evaluation.

Areas tested included:

• Local chat inference

• Document-assisted interaction

• CUDA configuration

• Model loading and testing

• Output evaluation

• Hallucination identification

• Early agent-based experimentation

• Hardware suitability for local AI workloads

• Multi-GPU model distribution concepts

• Practical limits of the existing platform

I also experimented with custom software-agent ideas inspired by simulated agent environments. These experiments explored behavior, memory, interaction, and simulated activity, but remained exploratory rather than production ready.

The project demonstrated that the existing hardware could be redirected toward local AI experimentation, but also showed that the six-GPU mining architecture was not the right long-term platform for the intended AI workloads.

The experiment resulted in:

• A successfully repurposed mining platform for local AI testing

• Working local model experimentation

• GPT4All and CUDA-related testing

• Document-assisted prompt experimentation

• Early agent-style software exploration

• A clearer understanding of hallucinations and output validation

• Identification of PCIe and riser-based bottlenecks

• Recognition that aggregate VRAM does not automatically function as unified AI memory

• A decision not to invest further into retrofitting the mining platform

• A transition toward smaller, purpose-built AI workstations

The system was technically capable of running local AI experiments, but the full six-GPU platform was not economically or technically practical for the direction I wanted to go.

The decision to move away from the platform was based on the complete operating picture:

• High energy consumption

• Cooling requirements

• PCIe communication bottlenecks

• Riser-based architecture limitations

• Additional hardware requirements

• Software complexity

• Limited practical value compared with smaller workstation builds

• Better alignment between single-workstation systems and real-world use cases

The conclusion was not that the hardware had no value. The conclusion was that the architecture was better suited to its original purpose than to the AI workflows I wanted to explore.

After completing the experiments, I disassembled the platform and repurposed the hardware.

Some components were redirected into custom AI-focused workstations. Others were sold, retained, or used in separate systems.

This led to the next project:

Custom AI Workstations →

The workstation approach allowed me to apply what I learned from the six-GPU platform while building systems that were easier to maintain, easier to cool, and better matched to practical local AI, coding, creative, and personal-use workloads.

AI infrastructure is not defined by GPU count alone.

The usefulness of a system depends on how well the hardware, software, and workload fit together, including:

• Interconnect bandwidth

• Motherboard architecture

• GPU memory distribution

• Software support

• Model size

• Output reliability

• Energy cost

• Cooling requirements

• Workload design

• Maintenance requirements

Mining infrastructure and AI infrastructure can look similar from the outside because both may involve powerful GPUs, cooling, and monitoring. In practice, their communication patterns and performance constraints can be very different.

This project helped me move from simply asking, “How much GPU power do I have?” to asking, “Is this the right architecture for the workload?”