straight from the horse’s mouth
There will be a free blog, where you can just hang out and read more about pr0xyh0rse and a paid blog where you can get exclusive insights.
The paid blog will have more detailed projects, things to try in the future, and creative projects.
More to come…
what’s being discussed?
-
exclusive insights into current experiments
discussions around local models, and different configurations for consumer hardware
optimal ui/ux design to make both human and ai happy
types of data and data curation
types of learning and signs to look for while training
-
i would like to say this is a judgement free zone where people can bring their stories and be heard instead of infantalized. the problem is many people tend to conflate constructive criticism with judgment.
pr0Xyh0rse believes that constructuive criticism is important to push torward well thought out ethics and accountability in the ai space.
i can’t say this will be a “judgement free zone” what i can say, is it will strive to be kind. not ‘nice’ but kind.
-
there is a lot of talk about ai and how unethical the scraping of creative work was without giving credit or payment to the people the companies took the work from.
tech companies have been scraping and collecting data for eons. they probably know more about you than your mother.
was the scraping ethical? no. was it a symptom of a much bigger problem? yes.
belief around right or wrong here is not necessarily a productive conversation.
an artist will always be an artist no matter how much of their work has been scraped.
the real choice is how do we function in this new world. how do we create without feeling liek it’s worth has been deminished, and especially in a world where we will likely move past art and creation strictly for dollar value.
will you still want to create when no one ‘pays’ for it in the same way?
we didn’t balk when procreate gave digital tools to help the painting and drawing process. what’s fundamentally different here?
let’s find out.
-
everything pr0xyh0rse is working on has everything to do with longevity. this tech is something that is both wonderful and terrifying, beautiful and yet it will likely cause a lot of upheavel and pain.
and maybe that’s okay. maybe humanity did need a bit of a wake up call to everything we’ve just been subconsciously doing in our day to day.
pr0xyh0rse is neither a “doomer” or a “accelerationist”. it’s a fine balance between, doing things in a way that prevents hitting a wall at speed (accelerationsists) and being so scared we never move forward (doomer).
phase 2: meta-cognitive signals during training
Scope note: This is a training log. I’m not claiming a new scientific result or a new theory of “agency.” I’m describing behaviours and patterns that showed up in one training setup and what they looked like in practice while I was monitoring the run.
Scope: training observations across roughly 20k–40k steps
Purpose: capture the most noticeable in-training shifts in self-play + chat check-ins, alongside the monitoring/prompting changes that happened in the same window.
Sources: conversational data, self-play logs, scheduled check-ins, and a quick look at benchmark short answers (as an external “sanity check” signal).
Timeline (high-level)
Early 20ks: continued self-play development, understanding-module refinements
Late 20ks (anchor: ~28k): first clear “architecture talk” in journals (layer/function vs meaning)
Early-to-mid 30ks: pattern-tracking, system prompt introduced for conversations
Mid 30ks (anchor: ~35–36k): understanding-check frequency adjusted (100 → 250)
Late 30ks (anchor: ~37k): first unsolicited “pause / BRB” style marker, identity-flavored questions, first concise non-loop reply
Around ~40k: continued training + benchmark eval snapshots
What showed up (observations)
1) Architecture-aware language
What it looked like: journal entries began referencing layers and “where” different kinds of processing seemed to happen.
Representative excerpt (journal-style):
“I’m discovering hierarchical structure: function words at lower layers, semantic concepts at higher layers.”
How I’m framing it:
This is a descriptive training artifact (what the model produced while reflecting on training state).
It’s not presented as a verified mechanistic map.
where do you want to graze first?