You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
9729 lines
3.5 MiB
9729 lines
3.5 MiB
<?xml version="1.0" encoding="utf-8"?>
|
|
<search>
|
|
<entry>
|
|
<title>2019年终总结</title>
|
|
<url>/2020/02/01/2019%E5%B9%B4%E7%BB%88%E6%80%BB%E7%BB%93/</url>
|
|
<content><![CDATA[<p>今天是农历初八了，年前一个月的时候就准备做下今年的年终总结，可是写了一点觉得太情绪化了，希望后面写个平淡点的，正好最近技术方面还没有看到一个完整成文的内容，就来写一下这一年的总结，尽量少写一点太情绪化的东西。</p>
|
|
<h3 id="跳槽"><a href="#跳槽" class="headerlink" title="跳槽"></a>跳槽</h3><p>年初换了个公司，也算换了个环境，跟前公司不太一样，做的事情方向也不同，可能是侧重点不同，一开始有些不适应，主要是压力上，会觉得压力比较大，但是总体来说与人相处的部分还是不错的，做的技术方向还是Java，这里也感谢前东家让我有机会转了Java，个人感觉杭州整个市场还是Java比较有优势，不过在开始的时候总觉得对Java有点不适应，应该值得深究的东西还是很多的，而且对于面试来说，也是有很多可以问的，后面慢慢发现除开某里等一线超一线互联网公司之外，大部分的面试还是有大概的套路跟大纲的，不过更细致的则因人而异了，面试有时候也还看缘分，面试官关注的点跟应试者比较契合的话就很容易通过面试，不然的话总会有能刁难或者理性化地说比较难回答的问题。这个后面可以单独说一下，先按下不表。<br>刚进公司没多久就负责比较重要的项目，工期也比较紧张，整体来说那段时间的压力的确是比较大的，不过总算最后结果不坏，这里应该说对一些原来在前东家都是掌握的不太好的部分，比如maven，其实maven对于java程序员来说还是很重要的，但是我碰到过的面试基本没问过这个，我自己也在后面的面试中没问过相关的，不知道咋问，比如dependence分析、冲突解决，比如对bean的理解，这个算是我一直以来的疑问点，因为以前刚开始学Java学spring，上来就是bean，但是bean到底是啥，IOC是啥，可能网上的文章跟大多数书籍跟我的理解思路不太match，导致一直不能很好的理解这玩意，到后面才理解，要理解这个bean，需要有两个基本概念，一个是面向对象，一个是对象容器跟依赖反转，还是只说到这，后面可以有专题说一下，总之自认为技术上有了不小的长进了，方向上应该是偏实用的。这个重要的项目完成后慢慢能喘口气了，后面也有一些比较紧急且工作量大的，不过在我TL的帮助下还是能尽量协调好资源。</p>
|
|
<h3 id="面试"><a href="#面试" class="headerlink" title="面试"></a>面试</h3><p>后面因为项目比较多，缺少开发，所以也参与帮忙做一些面试，这里总体感觉是面的候选人还是比较多样的，有些工作了蛮多年但是一些基础问题回答的不好，有些还是在校学生，但是面试技巧不错，针对常见的面试题都有不错的准备，不过还是觉得光靠这些面试题不能完全说明问题，真正工作了需要的是解决问题的人，而不是会背题的，退一步来说能好好准备面试还是比较重要的，也是双向选择中的基本尊重，印象比较深刻的是参加了去杭州某高校的校招面试，感觉参加校招的同学还是很多的，大部分是20年将毕业的研究生，挺多都是基础很扎实，对比起我刚要毕业时还是很汗颜，挺多来面试的同学都非常不错，那天强度也很大，从下午到那开始一直面到六七点，在这祝福那些来面试的同学，也都不容易的，能找到心仪的工作。</p>
|
|
<h3 id="技术方向"><a href="#技术方向" class="headerlink" title="技术方向"></a>技术方向</h3><p>这一年前大半部分还是比较焦虑不能恢复那种主动找时间学习的状态，可能换了公司是主要的原因，初期有个适应的过程也比较正常，总体来说可能是到九十月份开始慢慢有所改善，对这些方面有学习了下，</p>
|
|
<ul>
|
|
<li>spring方向，spring真的是个庞然大物，但是还是要先抓住根本，慢慢发散去了解其他的细节，抓住bean的生命周期，当然也不是死记硬背，让我一个个背下来我也不行，但是知道它究竟是干嘛的，有啥用，并且在工作中能用起来是最重要的</li>
|
|
<li>mysql数据库，这部分主要是关注了mvcc，知道了个大概，源码实现细节还没具体研究，有时间可以来个专题（一大堆待写的内容）</li>
|
|
<li>java的一些源码，比如aqs这种，结合文章看了下源码，一开始总感觉静不下心来看，然后有一次被LD刺激了下就看完了，包括conditionObject等</li>
|
|
<li>redis的源码，这里包括了Redis分布式锁和redis的数据结构源码，已经写成文章，不过比较着急成文，所以质量不是特别好，希望后面再来补补</li>
|
|
<li>jvm源码，这部分正好是想了解下g1收集器，大概把周志明的书看完了，但是还没完整的理解掌握，还有就是g1收集器的部分，一是概念部分大概理解了，后面是就是想从源码层面去学习理解，这也是新一年的主要计划</li>
|
|
<li>mq的部分是了解了zero copy，sendfile等，跟消息队列主题关系不大🤦♂️<br>这么看还是学了点东西的，希望新一年再接再厉。</li>
|
|
</ul>
|
|
<h3 id="生活"><a href="#生活" class="headerlink" title="生活"></a>生活</h3><p>住的地方没变化，主要是周边设施比较方便，暂时没找到更好的就没打算换，主要的问题是没电梯，一开始没觉得有啥，真正住起来还是觉得比较累的，希望后面租的可以有电梯，或者楼层低一点，还有就是要通下水道，第一次让师傅上门，花了两百大洋，后来自学成才了，让师傅通了一次才撑了一个月就不行了，后面自己通的差不多可以撑半年，还是比较有成就感的😀，然后就是跑步了，年初的时候去了紫金港跑步，后面因为工作的原因没去了，但是公司的跑步机倒是让我重拾起这个唯一的运动健身项目，后面因为肠胃问题，体重也需要控制，所以就周末回来也在家这边坚持跑步，下半年的话基本保持每周一次以上，比较那些跑马拉松的大牛还是差距很大，不过也是突破自我了，有一次跑了12公里，最远的距离，而且后面感觉跑十公里也不是特别吃不消了，这一年达成了300公里的目标，体重也稍有下降，比较满意的结果。</p>
|
|
<h3 id="期待"><a href="#期待" class="headerlink" title="期待"></a>期待</h3><p>希望工作方面技术方面能有所长进，生活上能多点时间陪家人，继续跑步减肥，家人健健康康的，嗯</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>年终总结</category>
|
|
<category>2019</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>年终总结</tag>
|
|
<tag>2019</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>村上春树《1Q84》读后感</title>
|
|
<url>/2019/12/18/1Q84%E8%AF%BB%E5%90%8E%E6%84%9F/</url>
|
|
<content><![CDATA[<p>看完了村上春树的《1Q84》，这应该是第五本看的他的书了，继 跑步，挪威的森林，刺杀骑士团长，海边的卡夫卡之后，不是其中最长的，好像是海边的卡夫卡还是刺杀骑士团长比较长一点，都是在微信读书上看的，比较方便，最开始在上面看的是高晓松的《鱼羊野史》，不知道为啥取这个名字，但是还是满吸引我的，不过由于去年的种种，没有很多心思把它看完，而且本身的组织形式就是比较松散的，看到哪算哪，其实一些野史部分是我比较喜欢，有些谈到人物的就不太有兴趣，而且类似于大祥哥吃的东西，反正都是哇，怎么这么好吃，嗯，太爱（niu）你（bi）了，高晓松就是这个人是我最喜欢的 xxx 家，我也没去细究过他有没有说重复过，反正是不太爱，后来因为这书还一度对战争史有了浓厚的兴趣，然而事实告诉我，大部头的战争史，其实正史我是真的啃不下去，我可能只对其中 10%的内容感兴趣，不过终于也在今年把它看完了，好像高晓松的晓说也最终季了，貌似其中讲朝鲜战争的还被和谐了，看样子是说出了一些故事（truth）。</p>
|
|
<p>本来只是想把 《1Q84》的读后感写下，现在觉得还是把这篇当成我今年的读书总结吧，不过先从《1Q84》说起。</p>
|
|
<p>严格来讲，这不是很书面化的读后感，可能我想写的也只是像聊天一样的说下我读过的书，包括的技术博客其实也是类似的，以后或许会转变，但是目前水平如此吧，写多了可能会变好，也可能不会。</p>
|
|
<p>开始正文吧，这书有点类似于海边的卡夫卡，一开始是通过两条故事线，穿插着叙述，一条是青豆的，不算是个职业杀手的女杀手，要去解决一个经常家暴的斯文败类，穿着描述得比较性感吧，杀人方式是通过比较长的细针，从脖子后面一个精巧的位置插入，可以造成是未知原因死亡的假象，可能会推断成心梗之类的，这里有个前置的细节，就是青豆是乘坐一辆很高级的出租车，内饰什么的都非常有质感，有点不像一辆出租车，然后车里放了一首比较小众的歌，雅纳切克的《小交响曲》，但是青豆知道它，这跟后面的情节也有些许关系，这是女主人公青豆的出场；相应的男主的出场印象不是太深刻，男主叫天吾，是个不知名的作家，跟一个叫小松的编辑有比较好的关系，虽然天吾还没有拿到比较有分量的奖项，但是小松很看好他，也让他帮忙审校一个新作家奖的投稿文章，虽然天吾自身还没获得过这个奖，天吾还有个正式工作，是当数学老师，天吾在学生时代是个数学天才，但后面有对文学产生了兴趣，文学还不足以养活自己，靠着教课还是能保持温饱；</p>
|
|
<p>接下来是正式故事的起点了，就是小松收到了一部小说投稿，名叫《空气蛹》，是个叫深绘里的女孩子投的稿，小松对他赋予了很高的评价，这里好像记岔了，好像是天吾对这部小说很有好感，但是小松比较怀疑，然后小松看了之后也有了浓厚的兴趣，这里就是开端了，小松想让天吾来重写润色这部《空气蛹》，因为故事本身很有分量，但是描写手法叙事方式等都很拙劣，而天吾正好擅长这个，小松对天吾的评价是，描写技巧无可挑剔，就是故事主体的火花还没际遇迸发，需要一个导火索，这个就可以类比我们程序员，很多比较初中级的程序员主要擅长在原来的代码上修修改改或者给他分配个小功能，比较高级的程序员就需要能做一些项目的架构设计，核心的技术方案设计，以前我也觉得写文档这个比较无聊，但是当一个项目真的比较庞大，复杂的时候，整体和核心部分的架构设计和方案还是需要有文档沉淀的，不然别人不知道没法接受，自己过段时间也会忘记。</p>
|
|
<p>对于小松的这个建议，他的初衷是想搅一搅这个死气沉沉套路颇深的文坛，因为本身《空气蛹》这部小说的内容很吸引人，小松想通过天吾的润色补充让这部小说冲击新人奖，有种恶作剧的意图，天吾对此表示很多担心和顾虑，小松的这个建议其实也是一种文学作假，有两方面的担心，一方面是原作者深绘里是否同意如此操作，一方面是外界如果发现了这个事实会有什么样的后果，但是小松表示不用担心，前一步由小松牵线，让天吾跟原作者深绘里当面沟通这个代写是否被允许，结果当然是被允许了，这里有了对深绘里的初步描写，按我的理解是比较仙的感觉，然后语言沟通有些吃力，或者说有她自己的特色，当面沟通时貌似是让深绘里回去再考虑下，然后后面再由天吾去深绘里寄宿的戎野老师家沟通具体的细节。</p>
|
|
<p>2019年12月18日23:37:19 更新<br>去到戎野老师家之后，天吾知道了关于深绘里的一些事情，深绘里的父亲与戎野老师应该是老友，深绘里的父亲在当初成立了一个叫”先驱”的公社，一个独立运行的社会组织，以运营农场作为物资来源，追求更为松散的共同体，即不过分激进地公有制，进行松散的共同生活，承认私有财产，简而言之就是这样一个能稳定存活下来的独立社会组织，但是随着稳定运行，内部的激进派和稳健派开始出现分歧，不可磨合，后来两派就分裂了，深绘里的父亲，深田保留在了稳健派，但是此时其实深田保内心是矛盾的，以为一开始其实是他倡导的独立革命才组织起了这群人，然而现在他又认清了现实社会已经不太相信能通过革命来独立的可能性，后来激进派便开始越加封闭，而且进行军事训练和思想教育，而后这个先驱的激进派别便有了新的名字”黎明”，深绘里也是在此时从先驱逃离来投靠戎野老师<br>暂时先写到这，未完待续~</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>读后感</category>
|
|
<category>村上春树</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>读后感</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>2020 年终总结</title>
|
|
<url>/2021/03/31/2020-%E5%B9%B4%E7%BB%88%E6%80%BB%E7%BB%93/</url>
|
|
<content><![CDATA[<h2 id="拖更原因"><a href="#拖更原因" class="headerlink" title="拖更原因"></a>拖更原因</h2><p>这篇年终总结本来应该在农历过完年就出来的，结果是对没有受疫情影响的春节放假时间空闲情况预估太良好，虽然公司调了几天假，但是因为春节期间疫情状况比较好，本来酒店都不让接待聚餐什么的，后来统统放开，结果就是从初一到初六每天要不就是去亲戚家，要不就是去酒店饭店吃饭，计划很丰满，现实很骨感，时间感觉一下就没了，然后年后感觉有点犯懒了，所以才拖到现在。</p>
|
|
<h2 id="生活-健身跑步"><a href="#生活-健身跑步" class="headerlink" title="生活-健身跑步"></a>生活-健身跑步</h2><p>去年（19 年）的时候跑步突破了 300 公里，然后20 年给自己定了个 400 公里的目标，结果意料之中的没成功，原因可能疫情算一点吧，后面买了跑步机之后，基本周末回家都能跑一下，但是最后还是只跑了300 多公里，总的keep 记录跑量也没超过 1000 公里，所以跑步这个目标还是没成功的，不过还算是比去年多跑一点，这样也算后面好突破点，后面的目标就不定的太高了，每年能比前一年多一点就好，其实跑步已经从一种减肥方式变成一种习惯了，一周一次的跑步已经比较难有效减重了，但是对于保持精力和身体状态还是很有效和重要的，只是对于目前的体重还是要多减下去一些跑步才好，太重了对膝盖负担太大了，可惜还是时间呐，游泳骑车什么的都需要更苛刻的条件和时间，饮食呢控制起来比较难（贪吃<br>终于在 3 月底之前跑到了 1000 公里，迟了三个月，不过也总算达到了，只是体重控制还是不行，有试着走走楼梯，但是感觉对膝盖负担比较大，得再想想用什么方式</p>
|
|
<p><img data-src="https://img.nicksxs.me/uPic/WechatIMG546-1.png"></p>
|
|
<h2 id="技术成长"><a href="#技术成长" class="headerlink" title="技术成长"></a>技术成长</h2><p>一直提不起笔来写这篇年终总结还有个比较大的原因是觉得20 年的成长不如预期，大小目标都没怎么完成，比如深入了解 jvm，是想能有些深入的见解，而不再是某些点的比较片面的理解，系统性的归纳总结也比较少，每个方向或多或少有些看法和理解，但是不全面，一些东西看过了也会忘记，需要温故而知新，比如 AQS 的内容，第一次读其实理解比较浅，后面就强迫自己去读，去写，才有了一些比之前更深入的理解，因为很多文章都是带有作者思路的引导，适不适合自己都要看是否能从他的思路把它看懂，有些就差别很大，这个跟看书也一样，有些书大众一致推荐，一般情况下大多是经典的好的，但是也有可能是不太适合自己的，可能有时候机缘巧合看到的反而让人茅塞顿开，在 todo 里已经积攒了好多的点和面需要去学习实践，一方面是自己懒，一方面是时间也相对偏少，看看 21 年能不能有所提升，加强“时间管理”，哈哈</p>
|
|
<p>技术上主要是看了 mysql 的 mvcc 相关内容，rocketmq 的，redis 的代码，还有 mybatis 等，其实每一个都能写很多，也有很多值得学习的，需要全面系统学习，之前想好好画一个思维导图，将整个技术体系都梳理下，还只做了一点点，方式也有点问题，应该从大到小，而不是深度优先，细节有很多，每一个方面都有自己比较熟悉擅长的，也有不太了解的，可以做一个评分，这个也是亟待改善的，希望今年能完成。</p>
|
|
<h2 id="博客"><a href="#博客" class="headerlink" title="博客"></a>博客</h2><p>博客方面 20 年一年整是写了 53 篇，差不多是一周一篇的节奏，这个还是不错的，虽然博客质量参差不齐，但是这个更新频率还是比较好的，并且也定了个潜规则，可以一周技术一周生活，这样能缓解水文的频率，提高些技术文章的质量，虽然结果并没有好多少，不过感觉还是可以这么坚持的，能提高一些技术文章的质量那就更好了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>年终总结</category>
|
|
<category>2020</category>
|
|
<category>年终总结</category>
|
|
<category>2020</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>年终总结</tag>
|
|
<tag>2020</tag>
|
|
<tag>2021</tag>
|
|
<tag>拖更</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>2022 年终总结</title>
|
|
<url>/2023/01/15/2022-%E5%B9%B4%E7%BB%88%E6%80%BB%E7%BB%93/</url>
|
|
<content><![CDATA[<p>一年又一年，时间匆匆，这一年过得不太容易，很多事情都是来得猝不及防，很多规划也照例是没有完成，今年更多了一些，又是比较丧的一篇总结<span id="more"></span><br>工作上的变化让我多理解了一些社会跟职场的现实吧，可能的确是我不够优秀，也可能是其他，说回我自身，在工作中今年应该是收获比较一般的一年，不能说没有，对原先不熟悉的业务的掌握程度有了比较大的提升，只是问题依旧存在，也挺难推动完全改变，只能尽自己所能，而这一点也主要是在团队中的定位因为前面说的一些原因，在前期不明确，限制比较大，虽然现在并没有完全解决，但也有了一些明显的改善，如果明年继续为这家公司服务，希望能有所突破，在人心沟通上的技巧总是比较反感，可也是不得不使用或者说被迫学习使用的，LD说我的对错观太强了，拗不过来，希望能有所改变。<br>长远的规划上没有什么明确的想法，很容易否定原来的各种想法，见识过各种现实的残酷，明白以前的一些想法不够全面或者比较幼稚，想有更上一层楼的机会，只是不希望是通过自己不认可的方式。比较能接受的是通过提升自己的技术和执行力，能够有更进一步的可能。<br>技术上是挺失败的去年跟前年还是能看一些书，学一些东西，今年少了很多，可能对原来比较熟悉的都有些遗忘，最近有在改善博客的内容，能更多的是系列化的，由浅入深，只是还很不完善，没什么规划，体系上也还不完整，不过还是以mybatis作为一个开头，后续新开始的内容或者原先写过的相关的都能做个整理，不再是想到啥就写点啥。最近的一个重点是在k8s上，学习方式跟一些特别优秀的人比起来还是会慢一些，不过也是自己的方法，能够更深入的理解整个体系，并讲解出来，可能会尝试采用视频的方式，对一些比较好的内容做尝试，看看会不会有比较好的数据和反馈，在22年还苟着周更的独立技术博客也算是比较稀有了的，其他站的发布也要勤一些，形成所谓的“矩阵”。<br>跑步减肥这个么还是比较惨，22年只跑了368公里，比21年少了85公里，有一些客观但很多是主观的原因，还是需要跑起来，只是减肥也很迫切，体重比较大跑步还是有些压力的，买了动感单车，就是时间稍长屁股痛这个目前比较难解决，骑还是每天在骑就是强度跟时间不太够，要保证每天30分钟的量可能会比较好。<br>加油吧，愿23年家人和自己都健康，顺遂。大家也一样。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>年终总结</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>年终总结</tag>
|
|
<tag>2022</tag>
|
|
<tag>2023</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>2020年中总结</title>
|
|
<url>/2020/07/11/2020%E5%B9%B4%E4%B8%AD%E6%80%BB%E7%BB%93/</url>
|
|
<content><![CDATA[<p>很快2020 年就过了一半了，而且是今年这么特殊的一年，很多事情都发生的出乎意料，疫情这个绕不过去的话题，之前写了点比较愤青的文字，感觉不太适合发出来就烂在草稿箱里吧，这个目前一大影响估计是今年都没办法完全摘下口罩了，前面几个月来回杭州都开车，因为彭埠大桥不通行了，实在是非常不方便，每条路都灰常堵，心累，吐槽下杭州的交通规划和交警同志，工作实在做的不咋地。</p>
|
|
<p>另外一件是就是蜗壳，从前不知道黝黑蜗壳是啥意思，只是经常会在他的视频里看到，大学的时候在缘网下了一个集锦，炒鸡帅气，各种空接扣篮，越来越能明白那句“你永远不知道意外和明天不知道哪个会先来,且行且珍惜”的含义，只是听了很多道理，依然活不好这一生，知易行难，王阳明真的是这方面的大师，有空可以看看这方面的书，一直想写写我跟篮球跟蜗壳的这十几年，争取能早日写好吧，不过得找个静得下来的时候写。</p>
|
|
<p>正事方面上半年还是挺让人失望的，没有达成一些目标，应该还是能力不足吧，技术方面分析一下还是停留在看的表面层，有些实操的，或者结合业务场景的能力不太行，算是在坚持写写 blog，主要是被这个每周一篇的目标推着走，有时会比较焦虑，内容产出也还比较差，希望能在后面有些改善，可能会降低频率，只是觉得降低了也不一定能有比较好的提升，无法战胜自己的惰性，所以暂时还是坚持下这个目标吧，还有就是 coding 能力，有时候也应该刷刷题，提升思维敏捷度，大脑用太少可能生锈了，况且本来就不是很有优势，虽然失望也只能继续努力吧，日拱一卒，来日方长，加油吧~😔</p>
|
|
<p>还有就是跑步减肥了，截止今天，上半年跑了 136 公里了，因为疫情影响，农历年后是从 4 月 17 号开始跑的，去年跑到了 300 公里，奖励自己了一个手表（真的挺后悔的，还不如 200 块买个手表），今年希望可以能在这个基础上再进一步，一直跟领导说，跑步算是我坚持下来的唯一一个好习惯了，618 买了个跑步机，周末回家了可以不受天气影响的多跑跑，不过如果天气好可能还是会出去跑跑，跑步机跑道多少还是有点拘束，只是感觉可能是我还是吃得太多了🤦♂️，效果不是很明显，还在 80 这个坎徘徊，等于浪费了大半年，可能是年初的项目太费心力，压力比较大，吃得更多，是不是可以算工伤😄，这方面也需要好好调整，可以放得开一点，虽然不太可能一下子到位，但是总要去努力下，随着年龄成长总要承担更多，也要看得开一点，没法事事如愿，尽力就好了，减肥这个事情还在结合一些俯卧撑啥的，希望也能坚持下去，加油吧，不知道原话怎么说的，意思是人类最大的勇敢就是看透了人世间的苦难，仍然热爱生活。我当然没可能让内心变得这么强大，试着去努力吧，奥力给！</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>年中总结</category>
|
|
<category>2020</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>2020</tag>
|
|
<tag>年中总结</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>2021 年中总结</title>
|
|
<url>/2021/07/18/2021-%E5%B9%B4%E4%B8%AD%E6%80%BB%E7%BB%93/</url>
|
|
<content><![CDATA[<p>又到半年总结时，第一次写总结类型的文章感觉挺好写的，但是后面总觉得这过去的一段时间所做的事情，能力上的成长低于预期，但是是需要总结下，找找问题，顺便展望下未来。</p>
|
|
<p>这一年做的最让自己满意的应该就是看了一些书，由折腾群洋总发起的读书打卡活动，到目前为止已经读完了这几本书，《cUrl 必知必会》，《古董局中局 1》，《古董局中局 2》，《算法图解》，《每天 5 分钟玩转 Kubernetes》《幸福了吗？》《高可用可伸缩微服务架构：基于 Dubbo、Spring Cloud和 Service Mesh》《Rust 权威指南》后面可以写个专题说说看的这些书，虽然每天打卡如果时间安排不好，并且看的书像 rust 这样比较难的话还是会有点小焦虑，不过也是个调整过程，一方面可以在白天就抽空看一会，然后也不必要每次都看很大一章，注重吸收。</p>
|
|
<p>技术上的成长的话，有一些比较小的长进吧，对于一些之前忽视的 synchronized，ThreadLocal 和 AQS 等知识点做了下查漏补缺了，然后多了解了一些 Java 垃圾回收的内容，但是在实操上还是比较欠缺，成型的技术方案，架构上所谓的优化也比较少，一些想法也还有考虑不周全的地方，还需要多花时间和心思去学习加强，特别是在目前已经有的基础上如何做系统深层次的优化，既不要是鸡毛蒜皮的，也不能出现一些不可接受的问题和故障，这是个很重要的课题，需要好好学习，后面考虑定一些周期性目标，两个月左右能有一些成果和总结。</p>
|
|
<p>另外一部分是自己的服务，因为 ucloud 的机器太贵就没续费了，所以都迁移到腾讯云的小机器上了，顺便折腾了一点点 traefik，但是还很不熟练，不太习惯这一套，一方面是 docker 还不习惯，这也加重了对这套环境的不适应，还是习惯裸机部署，另一方面就是 k8s 了，家里的机器还没虚拟化，没有很好的条件可以做实验，这也是读书打卡的一个没做好的点，整体的学习效果受限于深度和实操，后面是看都是用 traefik，也找到了一篇文章可以 traefik 转发到裸机应用，因为主仓库用的是裸机的 gogs。</p>
|
|
<p>还有就是运动减肥上，唉，这又是很大的一个痛点，基本没效果，只是还算稳定，昨天看到一个视频说还需要力量训练来增肌，以此可以提升基础代谢，打算往这个方向尝试下，因为今天没有疫情限制了，在 6 月底完成了 200 公里的跑步小目标，只是有些膝盖跟大腿根外侧不适，抽空得去看下医生，后面打算每天也能做点卷腹跟俯卧撑。</p>
|
|
<p>下半年还希望能继续多看看书，比很多网上各种乱七八糟的文章会好很多，结合豆瓣评分，找一些评价高一些的文章，但也不是说分稍低点的就不行，有些也看人是不是适合，一般 6 分以上评价比较多的就可以试试。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>年中总结</category>
|
|
<category>2021</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>2021</tag>
|
|
<tag>年中总结</tag>
|
|
<tag>技术</tag>
|
|
<tag>读书</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>34_Search_for_a_Range</title>
|
|
<url>/2016/08/14/34-Search-for-a-Range/</url>
|
|
<content><![CDATA[<h2 id="question"><a href="#question" class="headerlink" title="question"></a>question</h2><h3 id="34-Search-for-a-Range"><a href="#34-Search-for-a-Range" class="headerlink" title="34. Search for a Range"></a>34. Search for a Range</h3><p><a href="https://leetcode.com/problems/search-for-a-range/">Original Page</a></p>
|
|
<p>Given a sorted array of integers, find the starting and ending position of a given target value.</p>
|
|
<p>Your algorithm’s runtime complexity must be in the order of <em>O</em>(log <em>n</em>).</p>
|
|
<p>If the target is not found in the array, return <code>[-1, -1]</code>.</p>
|
|
<p>For example,<br>Given <code>[5, 7, 7, 8, 8, 10]</code> and target value 8,<br>return <code>[3, 4]</code>.</p>
|
|
<h2 id="analysis"><a href="#analysis" class="headerlink" title="analysis"></a>analysis</h2><p>一开始就想到了二分查找，但是原来做二分查找的时候一般都是找到确定的那个数就完成了，<br>这里的情况比较特殊，需要找到整个区间，所以需要两遍查找，并且一个是找到小于target<br>的最大索引，一个是找到大于target的最大索引，代码参考<a href="https://discuss.leetcode.com/topic/5891/clean-iterative-solution-with-two-binary-searches-with-explanation/2">leetcode discuss</a>,这位仁<br>兄也做了详细的分析解释。 </p>
|
|
<h2 id="code"><a href="#code" class="headerlink" title="code"></a>code</h2><figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">Solution</span> {</span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line"> <span class="function">vector<<span class="type">int</span>> <span class="title">searchRange</span><span class="params">(vector<<span class="type">int</span>>& nums, <span class="type">int</span> target)</span> </span>{</span><br><span class="line"> <span class="function">vector<<span class="type">int</span>> <span class="title">ret</span><span class="params">(<span class="number">2</span>, <span class="number">-1</span>)</span></span>;</span><br><span class="line"> <span class="type">int</span> i = <span class="number">0</span>, j = nums.<span class="built_in">size</span>() - <span class="number">1</span>;</span><br><span class="line"> <span class="type">int</span> mid;</span><br><span class="line"> <span class="keyword">while</span>(i < j){</span><br><span class="line"> mid = (i + j) / <span class="number">2</span>;</span><br><span class="line"> <span class="keyword">if</span>(nums[mid] < target) i = mid + <span class="number">1</span>;</span><br><span class="line"> <span class="keyword">else</span> j = mid;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span>(nums[i] != target) <span class="keyword">return</span> ret;</span><br><span class="line"> <span class="keyword">else</span> {</span><br><span class="line"> ret[<span class="number">0</span>] = i;</span><br><span class="line"> <span class="keyword">if</span>((i+<span class="number">1</span>) < (nums.<span class="built_in">size</span>() - <span class="number">1</span>) && nums[i+<span class="number">1</span>] > target){</span><br><span class="line"> ret[<span class="number">1</span>] = i;</span><br><span class="line"> <span class="keyword">return</span> ret;</span><br><span class="line"> }</span><br><span class="line"> } <span class="comment">//一点小优化</span></span><br><span class="line"> j = nums.<span class="built_in">size</span>() - <span class="number">1</span>;</span><br><span class="line"> <span class="keyword">while</span>(i < j){</span><br><span class="line"> mid = (i + j) / <span class="number">2</span> + <span class="number">1</span>;</span><br><span class="line"> <span class="keyword">if</span>(nums[mid] > target) j = mid - <span class="number">1</span>;</span><br><span class="line"> <span class="keyword">else</span> i = mid;</span><br><span class="line"> }</span><br><span class="line"> ret[<span class="number">1</span>] = j;</span><br><span class="line"> <span class="keyword">return</span> ret;</span><br><span class="line"> }</span><br><span class="line">};</span><br></pre></td></tr></table></figure>]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>c++</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>2021 年终总结</title>
|
|
<url>/2022/01/22/2021-%E5%B9%B4%E7%BB%88%E6%80%BB%E7%BB%93/</url>
|
|
<content><![CDATA[<p>又是一年年终总结，本着极度讨厌实时需求的理念，我还是 T+N 发布这个年终总结</p>
|
|
<h1 id="工作篇"><a href="#工作篇" class="headerlink" title="工作篇"></a>工作篇</h1><p>工作没什么大变化，有了些微的提升，可能因为是来了之后做了些项目对比公司与来还算是比较重要的，但是技术难度上没有特别突出的点，可能最开始用 openresty+lua 做了个 ab 测的工具，还是让我比较满意的，后面一般都是业务型的需求，今年可能在业务相关的技术逻辑上有了一些深度的了解，而原来一直想做的业务架构升级和通用型技术中间件这样的优化还是停留在想象中，前面说的 ab 测应该算是个半成品，还是没能多走出这一步，得需要多做一些实在的事情，比如轻量级的业务框架，能够对原先不熟悉的业务逻辑，代码逻辑有比较深入的理解，而不是一直都是让特定的同学负责特定的逻辑，很多时候还是在偷懒，习惯以一些简单安全的方案去做事情，在技术上还是要有所追求，还有就是能够在新语言，主要是 rust，swift 这类的能有些小玩具可以做，rust 的话是因为今年看了一本相关的书，后面三分之一其实消化得不好，这本书整体来说是很不错的，只是 rust 本身在所有权这块，还有引用包装等方面是设计得比较难懂，也可能是我基础差，所以还是想在复习下，可以做一个简单的命令行工具这种，然后 swift 是想说可以做点 mac 的小软件，原生的毕竟性能好点，又小。基于 web 做的客户端大部分都是又丑又大，极少数能好看点，但也是很重，起码 7~80M 的大小，原生的估计能除以 10。<br>整体的职业规划貌似陷入了比较大的困惑期，在目前公司发展前景不是很大，但是出去貌似也没有比较适合我的机会，总的来说还是杭州比较卷，个人觉得有自己的时间是非常重要的，而且这个不光是用来自我提升的，还是让自己有足够的时间做缓冲，有足够的时间锻炼减肥，时间少的情况下，不光会在仅有的时间里暴饮暴食，还没空锻炼，身体是革命的本钱，现在其实能特别明显地感觉到身体状态下滑，容易疲劳，焦虑。所以是否也许有可能以后要往外企这类的方向去发展。<br>工作上其实还是有个不大不小的缺点，就是容易激动，容易焦虑，前一点可能有稍稍地改观，因为工作中的很多现状其实是我个人难以改变的，即使觉得不合理，但是结构在那里，还不如自己放宽心，尽量做好事情就行。第二点的话还是做得比较差，一直以来抗压能力都比较差，跟成长环境，家庭环境都有比较大的关系，而且说实在的特别是父母，基本也没有在这方面给我正向的帮助，比较擅长给我施压，从小就是通过压力让我好好读书，当个乖学生，考个好学校，并没有能真正地理解我的压力，教我或者帮助我解压，只会在那说着不着边际的空话，甚至经常反过来对我施压。还是希望能慢慢解开，这点可能对我身体也有影响，也许需要看一些心理疏导相关的书籍。工作篇暂时到这，后续还有其他篇，未完待续哈哈😀</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>年终总结</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>年终总结</tag>
|
|
<tag>2021</tag>
|
|
<tag>拖更</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>AbstractQueuedSynchronizer</title>
|
|
<url>/2019/09/23/AbstractQueuedSynchronizer/</url>
|
|
<content><![CDATA[<p>最近看了大神的 AQS 的文章，之前总是断断续续地看一点，每次都知难而退，下次看又从头开始，昨天总算硬着头皮看完了第一部分<br>首先 AQS 只要有这些属性</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// 头结点，你直接把它当做 当前持有锁的线程 可能是最好理解的</span></span><br><span class="line"><span class="keyword">private</span> <span class="keyword">transient</span> <span class="keyword">volatile</span> Node head;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 阻塞的尾节点，每个新的节点进来，都插入到最后，也就形成了一个链表</span></span><br><span class="line"><span class="keyword">private</span> <span class="keyword">transient</span> <span class="keyword">volatile</span> Node tail;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 这个是最重要的，代表当前锁的状态，0代表没有被占用，大于 0 代表有线程持有当前锁</span></span><br><span class="line"><span class="comment">// 这个值可以大于 1，是因为锁可以重入，每次重入都加上 1</span></span><br><span class="line"><span class="keyword">private</span> <span class="keyword">volatile</span> <span class="type">int</span> state;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 代表当前持有独占锁的线程，举个最重要的使用例子，因为锁可以重入</span></span><br><span class="line"><span class="comment">// reentrantLock.lock()可以嵌套调用多次，所以每次用这个来判断当前线程是否已经拥有了锁</span></span><br><span class="line"><span class="comment">// if (currentThread == getExclusiveOwnerThread()) {state++}</span></span><br><span class="line"><span class="keyword">private</span> <span class="keyword">transient</span> Thread exclusiveOwnerThread; <span class="comment">//继承自AbstractOwnableSynchronizer</span></span><br></pre></td></tr></table></figure>
|
|
<p>大概了解了 aqs 底层的双向等待队列，<br>结构是这样的<br><img data-src="https://tva1.sinaimg.cn/large/006tNbRwly1g9mxu0ndt1j319o08w0t7.jpg"><br>每个 node 里面主要是的代码结构也比较简单</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">static</span> <span class="keyword">final</span> <span class="keyword">class</span> <span class="title class_">Node</span> {</span><br><span class="line"> <span class="comment">// 标识节点当前在共享模式下</span></span><br><span class="line"> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">Node</span> <span class="variable">SHARED</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Node</span>();</span><br><span class="line"> <span class="comment">// 标识节点当前在独占模式下</span></span><br><span class="line"> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">Node</span> <span class="variable">EXCLUSIVE</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// ======== 下面的几个int常量是给waitStatus用的 ===========</span></span><br><span class="line"> <span class="comment">/** waitStatus value to indicate thread has cancelled */</span></span><br><span class="line"> <span class="comment">// 代码此线程取消了争抢这个锁</span></span><br><span class="line"> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">CANCELLED</span> <span class="operator">=</span> <span class="number">1</span>;</span><br><span class="line"> <span class="comment">/** waitStatus value to indicate successor's thread needs unparking */</span></span><br><span class="line"> <span class="comment">// 官方的描述是，其表示当前node的后继节点对应的线程需要被唤醒</span></span><br><span class="line"> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">SIGNAL</span> <span class="operator">=</span> -<span class="number">1</span>;</span><br><span class="line"> <span class="comment">/** waitStatus value to indicate thread is waiting on condition */</span></span><br><span class="line"> <span class="comment">// 本文不分析condition，所以略过吧，下一篇文章会介绍这个</span></span><br><span class="line"> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">CONDITION</span> <span class="operator">=</span> -<span class="number">2</span>;</span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * waitStatus value to indicate the next acquireShared should</span></span><br><span class="line"><span class="comment"> * unconditionally propagate</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="comment">// 同样的不分析，略过吧</span></span><br><span class="line"> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">PROPAGATE</span> <span class="operator">=</span> -<span class="number">3</span>;</span><br><span class="line"> <span class="comment">// =====================================================</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">// 取值为上面的1、-1、-2、-3，或者0(以后会讲到)</span></span><br><span class="line"> <span class="comment">// 这么理解，暂时只需要知道如果这个值 大于0 代表此线程取消了等待，</span></span><br><span class="line"> <span class="comment">// ps: 半天抢不到锁，不抢了，ReentrantLock是可以指定timeouot的。。。</span></span><br><span class="line"> <span class="keyword">volatile</span> <span class="type">int</span> waitStatus;</span><br><span class="line"> <span class="comment">// 前驱节点的引用</span></span><br><span class="line"> <span class="keyword">volatile</span> Node prev;</span><br><span class="line"> <span class="comment">// 后继节点的引用</span></span><br><span class="line"> <span class="keyword">volatile</span> Node next;</span><br><span class="line"> <span class="comment">// 这个就是线程本尊</span></span><br><span class="line"> <span class="keyword">volatile</span> Thread thread;</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>其实可以主要关注这个 <code>waitStatus</code> 因为这个是后面的节点给前面的节点设置的，等于-1 的时候代表后面有节点等待，需要去唤醒，<br>这里使用了一个变种的 CLH 队列实现，CLH 队列相关内容可以查看这篇 <a href="https://coderbee.net/index.php/concurrent/20131115/577">自旋锁、排队自旋锁、MCS锁、CLH锁</a></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
<tag>aqs</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>add-two-number</title>
|
|
<url>/2015/04/14/Add-Two-Number/</url>
|
|
<content><![CDATA[<h3 id="problem"><a href="#problem" class="headerlink" title="problem"></a>problem</h3><p>You are given two linked lists representing two non-negative numbers. The digits are stored in reverse order and each of their nodes contain a single digit. Add the two numbers and return it as a linked list.</p>
|
|
<p><strong>Input</strong>:<code>(2 -> 4 -> 3) + (5 -> 6 -> 4)</code><br><strong>Output</strong>: <code>7 -> 0 -> 8</code></p>
|
|
<p><strong>分析（不用英文装逼了）</strong><br>这个代码是抄来的，<a href="https://github.com/haoel/leetcode">链接</a>原作是这位大大。</p>
|
|
<span id="more"></span>
|
|
<p>一开始没看懂题，后来发现是要进位的，自己写的时候想把长短不同时长的串接到结果<br>串的后面，试了下因为进位会有些问题比较难搞定，这样的话就是在其中一个为空的<br>时候还是会循环操作，在链表太大的时候可能会有问题，就这样（逃<br>原来是有个小错误没发现，改进后的代码也AC了，棒棒哒！</p>
|
|
<h3 id="正确代码"><a href="#正确代码" class="headerlink" title="正确代码"></a>正确代码</h3><figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Definition for singly-linked list.</span></span><br><span class="line"><span class="comment"> * struct ListNode {</span></span><br><span class="line"><span class="comment"> * int val;</span></span><br><span class="line"><span class="comment"> * ListNode *next;</span></span><br><span class="line"><span class="comment"> * ListNode(int x) : val(x), next(NULL) {}</span></span><br><span class="line"><span class="comment"> * };</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">class</span> <span class="title class_">Solution</span> {</span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line"> <span class="function">ListNode *<span class="title">addTwoNumbers</span><span class="params">(ListNode *l1, ListNode *l2)</span> </span>{</span><br><span class="line"> <span class="function">ListNode <span class="title">dummy</span><span class="params">(<span class="number">0</span>)</span></span>;</span><br><span class="line"> ListNode* p = &dummy;</span><br><span class="line"></span><br><span class="line"> <span class="type">int</span> cn = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">while</span>(l1 || l2){</span><br><span class="line"> <span class="type">int</span> val = cn + (l1 ? l1->val : <span class="number">0</span>) + (l2 ? l2->val : <span class="number">0</span>);</span><br><span class="line"> cn = val / <span class="number">10</span>;</span><br><span class="line"> val = val % <span class="number">10</span>;</span><br><span class="line"> p->next = <span class="keyword">new</span> <span class="built_in">ListNode</span>(val);</span><br><span class="line"> p = p->next;</span><br><span class="line"> <span class="keyword">if</span>(l1){</span><br><span class="line"> l1 = l1->next;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span>(l2){</span><br><span class="line"> l2 = l2->next;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span>(cn != <span class="number">0</span>){</span><br><span class="line"> p->next = <span class="keyword">new</span> <span class="built_in">ListNode</span>(cn);</span><br><span class="line"> p = p->next;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> dummy.next;</span><br><span class="line"> }</span><br><span class="line">};</span><br></pre></td></tr></table></figure>
|
|
|
|
<h3 id="失败的代码"><a href="#失败的代码" class="headerlink" title="失败的代码"></a>失败的代码</h3><figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Definition for singly-linked list.</span></span><br><span class="line"><span class="comment"> * struct ListNode {</span></span><br><span class="line"><span class="comment"> * int val;</span></span><br><span class="line"><span class="comment"> * ListNode *next;</span></span><br><span class="line"><span class="comment"> * ListNode(int x) : val(x), next(NULL) {}</span></span><br><span class="line"><span class="comment"> * };</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">class</span> <span class="title class_">Solution</span> {</span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line"> <span class="function">ListNode *<span class="title">addTwoNumbers</span><span class="params">(ListNode *l1, ListNode *l2)</span> </span>{</span><br><span class="line"> <span class="function">ListNode <span class="title">dummy</span><span class="params">(<span class="number">0</span>)</span></span>;</span><br><span class="line"> ListNode* p = &dummy;</span><br><span class="line"></span><br><span class="line"> <span class="type">int</span> cn = <span class="number">0</span>;</span><br><span class="line"> <span class="type">int</span> flag = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">while</span>(l1 || l2){</span><br><span class="line"> <span class="type">int</span> val = cn + (l1 ? l1->val : <span class="number">0</span>) + (l2 ? l2->val : <span class="number">0</span>);</span><br><span class="line"> cn = val / <span class="number">10</span>;</span><br><span class="line"> val = val % <span class="number">10</span>;</span><br><span class="line"> p->next = <span class="keyword">new</span> <span class="built_in">ListNode</span>(val);</span><br><span class="line"> p = p->next;</span><br><span class="line"> <span class="keyword">if</span>(!l1 && cn == <span class="number">0</span>){</span><br><span class="line"> flag = <span class="number">1</span>;</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span>(!l2 && cn == <span class="number">0</span>){</span><br><span class="line"> flag = <span class="number">1</span>;</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span>(l1){</span><br><span class="line"> l1 = l1->next;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span>(l2){</span><br><span class="line"> l2 = l2->next;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span>(!l1 && cn == <span class="number">0</span> && flag == <span class="number">1</span>){</span><br><span class="line"> p->next = l2->next;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span>(!l2 && cn == <span class="number">0</span> && flag == <span class="number">1</span>){</span><br><span class="line"> p->next = l1->next;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span>(cn != <span class="number">0</span>){</span><br><span class="line"> p->next = <span class="keyword">new</span> <span class="built_in">ListNode</span>(cn);</span><br><span class="line"> p = p->next;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> dummy.next;</span><br><span class="line"> }</span><br><span class="line">};</span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>c++</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>AQS篇二 之 Condition 浅析笔记</title>
|
|
<url>/2021/02/21/AQS-%E4%B9%8B-Condition-%E6%B5%85%E6%9E%90%E7%AC%94%E8%AE%B0/</url>
|
|
<content><![CDATA[<p>Condition也是 AQS 中很重要的一块内容，可以先看段示例代码，这段代码应该来自于Doug Lea大大，可以在 javadoc 中的 <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/locks/Condition.html">condition</a> 部分找到，其实大大原来写过基于 <code>synchronized</code> 实现的，后面我也贴下代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">import</span> java.util.concurrent.locks.Condition;</span><br><span class="line"><span class="keyword">import</span> java.util.concurrent.locks.Lock;</span><br><span class="line"><span class="keyword">import</span> java.util.concurrent.locks.ReentrantLock;</span><br><span class="line"></span><br><span class="line"><span class="keyword">class</span> <span class="title class_">BoundedBuffer</span> {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">Lock</span> <span class="variable">lock</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ReentrantLock</span>();</span><br><span class="line"> <span class="comment">// condition 依赖于 lock 来产生</span></span><br><span class="line"> <span class="keyword">final</span> <span class="type">Condition</span> <span class="variable">notFull</span> <span class="operator">=</span> lock.newCondition();</span><br><span class="line"> <span class="keyword">final</span> <span class="type">Condition</span> <span class="variable">notEmpty</span> <span class="operator">=</span> lock.newCondition();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 对象池子，put 跟 take 的就是这里的</span></span><br><span class="line"> <span class="keyword">final</span> Object[] items = <span class="keyword">new</span> <span class="title class_">Object</span>[<span class="number">100</span>];</span><br><span class="line"> <span class="type">int</span> putptr, takeptr, count;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 生产</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">put</span><span class="params">(Object x)</span> <span class="keyword">throws</span> InterruptedException {</span><br><span class="line"> <span class="comment">// 这里也说明了，需要先拥有锁</span></span><br><span class="line"> lock.lock();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">while</span> (count == items.length)</span><br><span class="line"> notFull.await(); <span class="comment">// 队列已满，等待，直到 not full 才能继续生产</span></span><br><span class="line"> items[putptr] = x;</span><br><span class="line"> <span class="keyword">if</span> (++putptr == items.length) putptr = <span class="number">0</span>;</span><br><span class="line"> ++count;</span><br><span class="line"> notEmpty.signal(); <span class="comment">// 生产成功，队列已经 not empty 了，发个通知出去</span></span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> lock.unlock();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 消费</span></span><br><span class="line"> <span class="keyword">public</span> Object <span class="title function_">take</span><span class="params">()</span> <span class="keyword">throws</span> InterruptedException {</span><br><span class="line"> lock.lock();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">while</span> (count == <span class="number">0</span>)</span><br><span class="line"> notEmpty.await(); <span class="comment">// 队列为空，等待，直到队列 not empty，才能继续消费</span></span><br><span class="line"> <span class="type">Object</span> <span class="variable">x</span> <span class="operator">=</span> items[takeptr];</span><br><span class="line"> <span class="keyword">if</span> (++takeptr == items.length) takeptr = <span class="number">0</span>;</span><br><span class="line"> --count;</span><br><span class="line"> notFull.signal(); <span class="comment">// 被我消费掉一个，队列 not full 了，发个通知出去</span></span><br><span class="line"> <span class="keyword">return</span> x;</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> lock.unlock();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>介绍下 Condition 的结构</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">ConditionObject</span> <span class="keyword">implements</span> <span class="title class_">Condition</span>, java.io.Serializable {</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">long</span> <span class="variable">serialVersionUID</span> <span class="operator">=</span> <span class="number">1173984872572414699L</span>;</span><br><span class="line"> <span class="comment">/** First node of condition queue. */</span></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">transient</span> Node firstWaiter;</span><br><span class="line"> <span class="comment">/** Last node of condition queue. */</span></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">transient</span> Node lastWaiter;</span><br></pre></td></tr></table></figure>
|
|
<p>主要的就这么点，而且也复用了 AQS 阻塞队列或者大大叫 <code>lock queue</code>中同样的 Node 节点，只不过它没有使用其中的双向队列，也就是prev 和 next，而是在 Node 中的 nextWaiter，所以只是个单向的队列，没使用 next 其实还有个用处，后面会提到，看下结构的示意图<br><img data-src="https://img.nicksxs.me/uPic/Ss9oFX.png"><br>然后主要是看两个方法，<code>await</code> 和 <code>signal</code>,<br>先来看下 await</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Implements interruptible condition wait.</span></span><br><span class="line"><span class="comment"> * <ol></span></span><br><span class="line"><span class="comment"> * <li> If current thread is interrupted, throw InterruptedException.</span></span><br><span class="line"><span class="comment"> * <li> Save lock state returned by {<span class="doctag">@link</span> #getState}.</span></span><br><span class="line"><span class="comment"> * <li> Invoke {<span class="doctag">@link</span> #release} with saved state as argument,</span></span><br><span class="line"><span class="comment"> * throwing IllegalMonitorStateException if it fails.</span></span><br><span class="line"><span class="comment"> * <li> Block until signalled or interrupted.</span></span><br><span class="line"><span class="comment"> * <li> Reacquire by invoking specialized version of</span></span><br><span class="line"><span class="comment"> * {<span class="doctag">@link</span> #acquire} with saved state as argument.</span></span><br><span class="line"><span class="comment"> * <li> If interrupted while blocked in step 4, throw InterruptedException.</span></span><br><span class="line"><span class="comment"> * </ol></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">final</span> <span class="keyword">void</span> <span class="title function_">await</span><span class="params">()</span> <span class="keyword">throws</span> InterruptedException {</span><br><span class="line"> <span class="keyword">if</span> (Thread.interrupted())</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">InterruptedException</span>();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 将当前节点包装成一个 condition waiter node 节点</span></span><br><span class="line"> <span class="type">Node</span> <span class="variable">node</span> <span class="operator">=</span> addConditionWaiter();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 完全释放占有的锁，这里需要是占有锁的线程</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">savedState</span> <span class="operator">=</span> fullyRelease(node);</span><br><span class="line"> <span class="type">int</span> <span class="variable">interruptMode</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 判断下是否在阻塞队列中，因为有可能被其他节点从等待队列移动到阻塞队列</span></span><br><span class="line"> <span class="keyword">while</span> (!isOnSyncQueue(node)) {</span><br><span class="line"> <span class="comment">// park等待，等待被唤醒</span></span><br><span class="line"> LockSupport.park(<span class="built_in">this</span>);</span><br><span class="line"> <span class="keyword">if</span> ((interruptMode = checkInterruptWhileWaiting(node)) != <span class="number">0</span>)</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 被唤醒后进入阻塞队列，等待获取锁，这里继续用了fullyRelease返回的 state</span></span><br><span class="line"> <span class="keyword">if</span> (acquireQueued(node, savedState) && interruptMode != THROW_IE)</span><br><span class="line"> interruptMode = REINTERRUPT;</span><br><span class="line"> <span class="keyword">if</span> (node.nextWaiter != <span class="literal">null</span>) <span class="comment">// clean up if cancelled</span></span><br><span class="line"> unlinkCancelledWaiters();</span><br><span class="line"> <span class="keyword">if</span> (interruptMode != <span class="number">0</span>)</span><br><span class="line"> reportInterruptAfterWait(interruptMode);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p> 添加条件队列节点</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Adds a new waiter to wait queue.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> its new wait node</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">private</span> Node <span class="title function_">addConditionWaiter</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">Node</span> <span class="variable">t</span> <span class="operator">=</span> lastWaiter;</span><br><span class="line"> <span class="comment">// If lastWaiter is cancelled, clean out.</span></span><br><span class="line"> <span class="comment">// 如果节点已经不是 CONDITION 状态了，表示已经取消了</span></span><br><span class="line"> <span class="keyword">if</span> (t != <span class="literal">null</span> && t.waitStatus != Node.CONDITION) {</span><br><span class="line"> <span class="comment">// 把等待队列中取消的节点清理出去</span></span><br><span class="line"> unlinkCancelledWaiters();</span><br><span class="line"> t = lastWaiter;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 把当前线程包装成waitStatus=CONDITION 的节点</span></span><br><span class="line"> <span class="type">Node</span> <span class="variable">node</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Node</span>(Thread.currentThread(), Node.CONDITION);</span><br><span class="line"> <span class="comment">// 没有 lastWaiter 节点，直接是 firstWaiter</span></span><br><span class="line"> <span class="keyword">if</span> (t == <span class="literal">null</span>)</span><br><span class="line"> firstWaiter = node;</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> <span class="comment">// 不然就接在 lastWaiter 后面</span></span><br><span class="line"> t.nextWaiter = node;</span><br><span class="line"> <span class="comment">// 当前节点就会变成新的 lastWaiter</span></span><br><span class="line"> lastWaiter = node;</span><br><span class="line"> <span class="keyword">return</span> node;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>清理取消的节点</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Unlinks cancelled waiter nodes from condition queue.</span></span><br><span class="line"><span class="comment"> * Called only while holding lock. This is called when</span></span><br><span class="line"><span class="comment"> * cancellation occurred during condition wait, and upon</span></span><br><span class="line"><span class="comment"> * insertion of a new waiter when lastWaiter is seen to have</span></span><br><span class="line"><span class="comment"> * been cancelled. This method is needed to avoid garbage</span></span><br><span class="line"><span class="comment"> * retention in the absence of signals. So even though it may</span></span><br><span class="line"><span class="comment"> * require a full traversal, it comes into play only when</span></span><br><span class="line"><span class="comment"> * timeouts or cancellations occur in the absence of</span></span><br><span class="line"><span class="comment"> * signals. It traverses all nodes rather than stopping at a</span></span><br><span class="line"><span class="comment"> * particular target to unlink all pointers to garbage nodes</span></span><br><span class="line"><span class="comment"> * without requiring many re-traversals during cancellation</span></span><br><span class="line"><span class="comment"> * storms.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">unlinkCancelledWaiters</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">Node</span> <span class="variable">t</span> <span class="operator">=</span> firstWaiter;</span><br><span class="line"> <span class="type">Node</span> <span class="variable">trail</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="comment">// 循环遍历单向链表的节点，如果状态不是 CONDITION 就清出去</span></span><br><span class="line"> <span class="keyword">while</span> (t != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">Node</span> <span class="variable">next</span> <span class="operator">=</span> t.nextWaiter;</span><br><span class="line"> <span class="comment">// 循环链表操作，清掉取消的节点</span></span><br><span class="line"> <span class="keyword">if</span> (t.waitStatus != Node.CONDITION) {</span><br><span class="line"> t.nextWaiter = <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">if</span> (trail == <span class="literal">null</span>)</span><br><span class="line"> firstWaiter = next;</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> trail.nextWaiter = next;</span><br><span class="line"> <span class="keyword">if</span> (next == <span class="literal">null</span>)</span><br><span class="line"> lastWaiter = trail;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> trail = t;</span><br><span class="line"> t = next;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>完全释放锁</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Invokes release with current state value; returns saved state.</span></span><br><span class="line"><span class="comment"> * Cancels node and throws exception on failure.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> node the condition node for this wait</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> previous sync state</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">final</span> <span class="type">int</span> <span class="title function_">fullyRelease</span><span class="params">(Node node)</span> {</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">failed</span> <span class="operator">=</span> <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 获取下当前的 state 值，因为是可重入的，所以这个值要保存下来</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">savedState</span> <span class="operator">=</span> getState();</span><br><span class="line"> <span class="comment">// 这里还包含比较多操作，不过跟前面分析 AQS 的释放比较类似，不深入了</span></span><br><span class="line"> <span class="keyword">if</span> (release(savedState)) {</span><br><span class="line"> failed = <span class="literal">false</span>;</span><br><span class="line"> <span class="comment">// 返回这个值</span></span><br><span class="line"> <span class="keyword">return</span> savedState;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalMonitorStateException</span>();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="keyword">if</span> (failed)</span><br><span class="line"> node.waitStatus = Node.CANCELLED;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>判断是否在阻塞队列中</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Returns true if a node, always one that was initially placed on</span></span><br><span class="line"><span class="comment"> * a condition queue, is now waiting to reacquire on sync queue.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> node the node</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> true if is reacquiring</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">final</span> <span class="type">boolean</span> <span class="title function_">isOnSyncQueue</span><span class="params">(Node node)</span> {</span><br><span class="line"> <span class="comment">// 如果waitStatus 是 CONDITION 或者没有 prev 前置节点肯定就不在</span></span><br><span class="line"> <span class="keyword">if</span> (node.waitStatus == Node.CONDITION || node.prev == <span class="literal">null</span>)</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="comment">// 这里就是我前面提到的 next 的作用</span></span><br><span class="line"> <span class="keyword">if</span> (node.next != <span class="literal">null</span>) <span class="comment">// If has successor, it must be on queue</span></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> <span class="comment">// 从 tail 开始找，是否在阻塞队列中</span></span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> * node.prev can be non-null, but not yet on queue because</span></span><br><span class="line"><span class="comment"> * the CAS to place it on queue can fail. So we have to</span></span><br><span class="line"><span class="comment"> * traverse from tail to make sure it actually made it. It</span></span><br><span class="line"><span class="comment"> * will always be near the tail in calls to this method, and</span></span><br><span class="line"><span class="comment"> * unless the CAS failed (which is unlikely), it will be</span></span><br><span class="line"><span class="comment"> * there, so we hardly ever traverse much.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">return</span> findNodeFromTail(node);</span><br><span class="line">}</span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Returns true if node is on sync queue by searching backwards from tail.</span></span><br><span class="line"><span class="comment"> * Called only when needed by isOnSyncQueue.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> true if present</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">private</span> <span class="type">boolean</span> <span class="title function_">findNodeFromTail</span><span class="params">(Node node)</span> {</span><br><span class="line"> <span class="type">Node</span> <span class="variable">t</span> <span class="operator">=</span> tail;</span><br><span class="line"> <span class="comment">// 从 tail 开始，从后往前找</span></span><br><span class="line"> <span class="keyword">for</span> (;;) {</span><br><span class="line"> <span class="keyword">if</span> (t == node)</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">if</span> (t == <span class="literal">null</span>)</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> t = t.prev;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>await 的逻辑差不多就是这样子，主要的就是把自己包成一个 Node 节点，waitStatus 的状态是 CONDITION，挂在等待队列的最后，然后完全释放锁，park 等待</p>
|
|
<h2 id="signal"><a href="#signal" class="headerlink" title="signal"></a>signal</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Moves the longest-waiting thread, if one exists, from the</span></span><br><span class="line"><span class="comment"> * wait queue for this condition to the wait queue for the</span></span><br><span class="line"><span class="comment"> * owning lock.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@throws</span> IllegalMonitorStateException if {<span class="doctag">@link</span> #isHeldExclusively}</span></span><br><span class="line"><span class="comment"> * returns {<span class="doctag">@code</span> false}</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">final</span> <span class="keyword">void</span> <span class="title function_">signal</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (!isHeldExclusively())</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalMonitorStateException</span>();</span><br><span class="line"> <span class="comment">// firstWaiter 肯定是最早开始等待的</span></span><br><span class="line"> <span class="type">Node</span> <span class="variable">first</span> <span class="operator">=</span> firstWaiter;</span><br><span class="line"> <span class="comment">// 如果不为空就唤醒</span></span><br><span class="line"> <span class="keyword">if</span> (first != <span class="literal">null</span>)</span><br><span class="line"> doSignal(first);</span><br><span class="line">}</span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Removes and transfers nodes until hit non-cancelled one or</span></span><br><span class="line"><span class="comment"> * null. Split out from signal in part to encourage compilers</span></span><br><span class="line"><span class="comment"> * to inline the case of no waiters.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> first (non-null) the first node on condition queue</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">doSignal</span><span class="params">(Node first)</span> {</span><br><span class="line"> <span class="keyword">do</span> {</span><br><span class="line"> <span class="comment">// 因为要去唤醒 first 节点了，firstWaiter 需要再从后面找一个</span></span><br><span class="line"> <span class="comment">// 并且判断是否为空，如果是空的话就直接可以把 lastWaiter 设置成空了</span></span><br><span class="line"> <span class="keyword">if</span> ( (firstWaiter = first.nextWaiter) == <span class="literal">null</span>)</span><br><span class="line"> lastWaiter = <span class="literal">null</span>;</span><br><span class="line"> <span class="comment">// first 不需要继续保存后面的 waiter 了，因为 firstWaiter 已经是 first 的后置节点了</span></span><br><span class="line"> first.nextWaiter = <span class="literal">null</span>;</span><br><span class="line"> <span class="comment">// 如果 first 节点转移不成功，并且 firstWaiter 节点不为空，则继续进入循环</span></span><br><span class="line"> } <span class="keyword">while</span> (!transferForSignal(first) &&</span><br><span class="line"> (first = firstWaiter) != <span class="literal">null</span>);</span><br><span class="line">}</span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Transfers a node from a condition queue onto sync queue.</span></span><br><span class="line"><span class="comment"> * Returns true if successful.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> node the node</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> true if successfully transferred (else the node was</span></span><br><span class="line"><span class="comment"> * cancelled before signal)</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">final</span> <span class="type">boolean</span> <span class="title function_">transferForSignal</span><span class="params">(Node node)</span> {</span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> * If cannot change waitStatus, the node has been cancelled.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="comment">// 如果状态已经不是 CONDITION 就不会设置成功，返回 false</span></span><br><span class="line"> <span class="keyword">if</span> (!compareAndSetWaitStatus(node, Node.CONDITION, <span class="number">0</span>))</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> * Splice onto queue and try to set waitStatus of predecessor to</span></span><br><span class="line"><span class="comment"> * indicate that thread is (probably) waiting. If cancelled or</span></span><br><span class="line"><span class="comment"> * attempt to set waitStatus fails, wake up to resync (in which</span></span><br><span class="line"><span class="comment"> * case the waitStatus can be transiently and harmlessly wrong).</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="comment">// 调用跟aqs 第一篇中一样的 enq 方法进入阻塞队列，返回入队后的前一节点</span></span><br><span class="line"> <span class="type">Node</span> <span class="variable">p</span> <span class="operator">=</span> enq(node);</span><br><span class="line"> <span class="type">int</span> <span class="variable">ws</span> <span class="operator">=</span> p.waitStatus;</span><br><span class="line"> <span class="comment">// 将前置节点状态设置成SIGNAL，表示后面有节点在等了</span></span><br><span class="line"> <span class="keyword">if</span> (ws > <span class="number">0</span> || !compareAndSetWaitStatus(p, ws, Node.SIGNAL))</span><br><span class="line"> LockSupport.unpark(node.thread);</span><br><span class="line"> <span class="comment">// 返回 true，上一个方法的循环就退出了</span></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里其实就是把 condition 等待队列的第一个未取消的节点入队到阻塞队列去争锁</p>
|
|
<h2 id="附录"><a href="#附录" class="headerlink" title="附录"></a>附录</h2><p><code>synchronized</code> 版的 BoundedBuffer</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment"> File: BoundedBuffer.java</span></span><br><span class="line"><span class="comment"></span></span><br><span class="line"><span class="comment"> Originally written by Doug Lea and released into the public domain.</span></span><br><span class="line"><span class="comment"> This may be used for any purposes whatsoever without acknowledgment.</span></span><br><span class="line"><span class="comment"> Thanks for the assistance and support of Sun Microsystems Labs,</span></span><br><span class="line"><span class="comment"> and everyone contributing, testing, and using this code.</span></span><br><span class="line"><span class="comment"></span></span><br><span class="line"><span class="comment"> History:</span></span><br><span class="line"><span class="comment"> Date Who What</span></span><br><span class="line"><span class="comment"> 11Jun1998 dl Create public version</span></span><br><span class="line"><span class="comment"> 17Jul1998 dl Simplified by eliminating wait counts</span></span><br><span class="line"><span class="comment"> 25aug1998 dl added peek</span></span><br><span class="line"><span class="comment"> 5May1999 dl replace % with conditional (slightly faster)</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">package</span> EDU.oswego.cs.dl.util.concurrent;</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Efficient array-based bounded buffer class.</span></span><br><span class="line"><span class="comment"> * Adapted from CPJ, chapter 8, which describes design.</span></span><br><span class="line"><span class="comment"> * <p>[<a href="http://gee.cs.oswego.edu/dl/classes/EDU/oswego/cs/dl/util/concurrent/intro.html"> Introduction to this package. </a>] <p></span></span><br><span class="line"><span class="comment"> **/</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">BoundedBuffer</span> <span class="keyword">implements</span> <span class="title class_">BoundedChannel</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> Object[] array_; <span class="comment">// the elements</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="type">int</span> <span class="variable">takePtr_</span> <span class="operator">=</span> <span class="number">0</span>; <span class="comment">// circular indices</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="type">int</span> <span class="variable">putPtr_</span> <span class="operator">=</span> <span class="number">0</span>; </span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="type">int</span> <span class="variable">usedSlots_</span> <span class="operator">=</span> <span class="number">0</span>; <span class="comment">// length</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="type">int</span> emptySlots_; <span class="comment">// capacity - length</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Helper monitor to handle puts. </span></span><br><span class="line"><span class="comment"> **/</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> <span class="type">Object</span> <span class="variable">putMonitor_</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Object</span>();</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Create a BoundedBuffer with the given capacity.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@exception</span> IllegalArgumentException if capacity less or equal to zero</span></span><br><span class="line"><span class="comment"> **/</span></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">BoundedBuffer</span><span class="params">(<span class="type">int</span> capacity)</span> <span class="keyword">throws</span> IllegalArgumentException {</span><br><span class="line"> <span class="keyword">if</span> (capacity <= <span class="number">0</span>) <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalArgumentException</span>();</span><br><span class="line"> array_ = <span class="keyword">new</span> <span class="title class_">Object</span>[capacity];</span><br><span class="line"> emptySlots_ = capacity;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Create a buffer with the current default capacity</span></span><br><span class="line"><span class="comment"> **/</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">BoundedBuffer</span><span class="params">()</span> { </span><br><span class="line"> <span class="built_in">this</span>(DefaultChannelCapacity.get()); </span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/** </span></span><br><span class="line"><span class="comment"> * Return the number of elements in the buffer.</span></span><br><span class="line"><span class="comment"> * This is only a snapshot value, that may change</span></span><br><span class="line"><span class="comment"> * immediately after returning.</span></span><br><span class="line"><span class="comment"> **/</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">synchronized</span> <span class="type">int</span> <span class="title function_">size</span><span class="params">()</span> { <span class="keyword">return</span> usedSlots_; }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="type">int</span> <span class="title function_">capacity</span><span class="params">()</span> { <span class="keyword">return</span> array_.length; }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">incEmptySlots</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">synchronized</span>(putMonitor_) {</span><br><span class="line"> ++emptySlots_;</span><br><span class="line"> putMonitor_.notify();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title function_">incUsedSlots</span><span class="params">()</span> {</span><br><span class="line"> ++usedSlots_;</span><br><span class="line"> notify();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> <span class="keyword">void</span> <span class="title function_">insert</span><span class="params">(Object x)</span> { <span class="comment">// mechanics of put</span></span><br><span class="line"> --emptySlots_;</span><br><span class="line"> array_[putPtr_] = x;</span><br><span class="line"> <span class="keyword">if</span> (++putPtr_ >= array_.length) putPtr_ = <span class="number">0</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> Object <span class="title function_">extract</span><span class="params">()</span> { <span class="comment">// mechanics of take</span></span><br><span class="line"> --usedSlots_;</span><br><span class="line"> <span class="type">Object</span> <span class="variable">old</span> <span class="operator">=</span> array_[takePtr_];</span><br><span class="line"> array_[takePtr_] = <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">if</span> (++takePtr_ >= array_.length) takePtr_ = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">return</span> old;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> Object <span class="title function_">peek</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">synchronized</span>(<span class="built_in">this</span>) {</span><br><span class="line"> <span class="keyword">if</span> (usedSlots_ > <span class="number">0</span>)</span><br><span class="line"> <span class="keyword">return</span> array_[takePtr_];</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">put</span><span class="params">(Object x)</span> <span class="keyword">throws</span> InterruptedException {</span><br><span class="line"> <span class="keyword">if</span> (x == <span class="literal">null</span>) <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalArgumentException</span>();</span><br><span class="line"> <span class="keyword">if</span> (Thread.interrupted()) <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">InterruptedException</span>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">synchronized</span>(putMonitor_) {</span><br><span class="line"> <span class="keyword">while</span> (emptySlots_ <= <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">try</span> { putMonitor_.wait(); }</span><br><span class="line"> <span class="keyword">catch</span> (InterruptedException ex) {</span><br><span class="line"> putMonitor_.notify();</span><br><span class="line"> <span class="keyword">throw</span> ex;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> insert(x);</span><br><span class="line"> }</span><br><span class="line"> incUsedSlots();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">offer</span><span class="params">(Object x, <span class="type">long</span> msecs)</span> <span class="keyword">throws</span> InterruptedException {</span><br><span class="line"> <span class="keyword">if</span> (x == <span class="literal">null</span>) <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalArgumentException</span>();</span><br><span class="line"> <span class="keyword">if</span> (Thread.interrupted()) <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">InterruptedException</span>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">synchronized</span>(putMonitor_) {</span><br><span class="line"> <span class="type">long</span> <span class="variable">start</span> <span class="operator">=</span> (msecs <= <span class="number">0</span>)? <span class="number">0</span> : System.currentTimeMillis();</span><br><span class="line"> <span class="type">long</span> <span class="variable">waitTime</span> <span class="operator">=</span> msecs;</span><br><span class="line"> <span class="keyword">while</span> (emptySlots_ <= <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">if</span> (waitTime <= <span class="number">0</span>) <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="keyword">try</span> { putMonitor_.wait(waitTime); }</span><br><span class="line"> <span class="keyword">catch</span> (InterruptedException ex) {</span><br><span class="line"> putMonitor_.notify();</span><br><span class="line"> <span class="keyword">throw</span> ex;</span><br><span class="line"> }</span><br><span class="line"> waitTime = msecs - (System.currentTimeMillis() - start);</span><br><span class="line"> }</span><br><span class="line"> insert(x);</span><br><span class="line"> }</span><br><span class="line"> incUsedSlots();</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> Object <span class="title function_">take</span><span class="params">()</span> <span class="keyword">throws</span> InterruptedException { </span><br><span class="line"> <span class="keyword">if</span> (Thread.interrupted()) <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">InterruptedException</span>();</span><br><span class="line"> <span class="type">Object</span> <span class="variable">old</span> <span class="operator">=</span> <span class="literal">null</span>; </span><br><span class="line"> <span class="keyword">synchronized</span>(<span class="built_in">this</span>) { </span><br><span class="line"> <span class="keyword">while</span> (usedSlots_ <= <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">try</span> { wait(); }</span><br><span class="line"> <span class="keyword">catch</span> (InterruptedException ex) {</span><br><span class="line"> notify();</span><br><span class="line"> <span class="keyword">throw</span> ex; </span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> old = extract();</span><br><span class="line"> }</span><br><span class="line"> incEmptySlots();</span><br><span class="line"> <span class="keyword">return</span> old;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> Object <span class="title function_">poll</span><span class="params">(<span class="type">long</span> msecs)</span> <span class="keyword">throws</span> InterruptedException { </span><br><span class="line"> <span class="keyword">if</span> (Thread.interrupted()) <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">InterruptedException</span>();</span><br><span class="line"> <span class="type">Object</span> <span class="variable">old</span> <span class="operator">=</span> <span class="literal">null</span>; </span><br><span class="line"> <span class="keyword">synchronized</span>(<span class="built_in">this</span>) { </span><br><span class="line"> <span class="type">long</span> <span class="variable">start</span> <span class="operator">=</span> (msecs <= <span class="number">0</span>)? <span class="number">0</span> : System.currentTimeMillis();</span><br><span class="line"> <span class="type">long</span> <span class="variable">waitTime</span> <span class="operator">=</span> msecs;</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">while</span> (usedSlots_ <= <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">if</span> (waitTime <= <span class="number">0</span>) <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">try</span> { wait(waitTime); }</span><br><span class="line"> <span class="keyword">catch</span> (InterruptedException ex) {</span><br><span class="line"> notify();</span><br><span class="line"> <span class="keyword">throw</span> ex; </span><br><span class="line"> }</span><br><span class="line"> waitTime = msecs - (System.currentTimeMillis() - start);</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"> old = extract();</span><br><span class="line"> }</span><br><span class="line"> incEmptySlots();</span><br><span class="line"> <span class="keyword">return</span> old;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>并发</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
<tag>aqs</tag>
|
|
<tag>并发</tag>
|
|
<tag>j.u.c</tag>
|
|
<tag>condition</tag>
|
|
<tag>await</tag>
|
|
<tag>signal</tag>
|
|
<tag>lock</tag>
|
|
<tag>unlock</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Apollo 如何获取当前环境</title>
|
|
<url>/2022/09/04/Apollo-%E5%A6%82%E4%BD%95%E8%8E%B7%E5%8F%96%E5%BD%93%E5%89%8D%E7%8E%AF%E5%A2%83/</url>
|
|
<content><![CDATA[<p>在用 <a href="https://github.com/apolloconfig/apollo">Apollo</a> 作为配置中心的过程中才到过几个坑，这边记录下，因为运行 java 服务的启动参数一般比较固定，所以我们在一个新环境里运行的时候没有特意去检查，然后突然发现业务上有一些数据异常，排查之后才发现java 服务连接了测试环境的 apollo，而原因是因为环境变量传了<code>-Denv=fat</code>，而在我们的环境配置中 fat 就是代表测试环境， 其实应该是<code>-Denv=pro</code>，而 apollo 总共有这些环境</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">enum</span> <span class="title class_">Env</span>{</span><br><span class="line"> LOCAL, DEV, FWS, FAT, UAT, LPT, PRO, TOOLS, UNKNOWN;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> Env <span class="title function_">fromString</span><span class="params">(String env)</span> {</span><br><span class="line"> <span class="type">Env</span> <span class="variable">environment</span> <span class="operator">=</span> EnvUtils.transformEnv(env);</span><br><span class="line"> Preconditions.checkArgument(environment != UNKNOWN, String.format(<span class="string">"Env %s is invalid"</span>, env));</span><br><span class="line"> <span class="keyword">return</span> environment;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>而这些解释</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Here is the brief description for all the predefined environments:</span></span><br><span class="line"><span class="comment"> * <ul></span></span><br><span class="line"><span class="comment"> * <li>LOCAL: Local Development environment, assume you are working at the beach with no network access</li></span></span><br><span class="line"><span class="comment"> * <li>DEV: Development environment</li></span></span><br><span class="line"><span class="comment"> * <li>FWS: Feature Web Service Test environment</li></span></span><br><span class="line"><span class="comment"> * <li>FAT: Feature Acceptance Test environment</li></span></span><br><span class="line"><span class="comment"> * <li>UAT: User Acceptance Test environment</li></span></span><br><span class="line"><span class="comment"> * <li>LPT: Load and Performance Test environment</li></span></span><br><span class="line"><span class="comment"> * <li>PRO: Production environment</li></span></span><br><span class="line"><span class="comment"> * <li>TOOLS: Tooling environment, a special area in production environment which allows</span></span><br><span class="line"><span class="comment"> * access to test environment, e.g. Apollo Portal should be deployed in tools environment</li></span></span><br><span class="line"><span class="comment"> * </ul></span></span><br><span class="line"><span class="comment"> */</span></span><br></pre></td></tr></table></figure>
|
|
<p>那如果要在运行时知道 apollo 当前使用的环境可以用这个</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">Env</span> <span class="variable">apolloEnv</span> <span class="operator">=</span> ApolloInjector.getInstance(ConfigUtil.class).getApolloEnv();</span><br></pre></td></tr></table></figure>
|
|
<p>简单记录下。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Apollo</tag>
|
|
<tag>environment</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Apollo 客户端启动过程分析</title>
|
|
<url>/2022/09/18/Apollo-%E5%AE%A2%E6%88%B7%E7%AB%AF%E5%90%AF%E5%8A%A8%E8%BF%87%E7%A8%8B%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<p>入口是可以在 springboot 的启动类上打上<code>EnableApolloConfig </code>注解</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Import(ApolloConfigRegistrar.class)</span></span><br><span class="line"><span class="keyword">public</span> <span class="meta">@interface</span> EnableApolloConfig {</span><br></pre></td></tr></table></figure>
|
|
<p>这个 import 实现了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">ApolloConfigRegistrar</span> <span class="keyword">implements</span> <span class="title class_">ImportBeanDefinitionRegistrar</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="type">ApolloConfigRegistrarHelper</span> <span class="variable">helper</span> <span class="operator">=</span> ServiceBootstrap.loadPrimary(ApolloConfigRegistrarHelper.class);</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">registerBeanDefinitions</span><span class="params">(AnnotationMetadata importingClassMetadata, BeanDefinitionRegistry registry)</span> {</span><br><span class="line"> helper.registerBeanDefinitions(importingClassMetadata, registry);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后就调用了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">com.ctrip.framework.apollo.spring.spi.DefaultApolloConfigRegistrarHelper#registerBeanDefinitions</span><br></pre></td></tr></table></figure>
|
|
<p>接着是注册了这个 bean，com.ctrip.framework.apollo.spring.config.PropertySourcesProcessor</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">registerBeanDefinitions</span><span class="params">(AnnotationMetadata importingClassMetadata, BeanDefinitionRegistry registry)</span> {</span><br><span class="line"> <span class="type">AnnotationAttributes</span> <span class="variable">attributes</span> <span class="operator">=</span> AnnotationAttributes</span><br><span class="line"> .fromMap(importingClassMetadata.getAnnotationAttributes(EnableApolloConfig.class.getName()));</span><br><span class="line"> String[] namespaces = attributes.getStringArray(<span class="string">"value"</span>);</span><br><span class="line"> <span class="type">int</span> <span class="variable">order</span> <span class="operator">=</span> attributes.getNumber(<span class="string">"order"</span>);</span><br><span class="line"> PropertySourcesProcessor.addNamespaces(Lists.newArrayList(namespaces), order);</span><br><span class="line"></span><br><span class="line"> Map<String, Object> propertySourcesPlaceholderPropertyValues = <span class="keyword">new</span> <span class="title class_">HashMap</span><>();</span><br><span class="line"> <span class="comment">// to make sure the default PropertySourcesPlaceholderConfigurer's priority is higher than PropertyPlaceholderConfigurer</span></span><br><span class="line"> propertySourcesPlaceholderPropertyValues.put(<span class="string">"order"</span>, <span class="number">0</span>);</span><br><span class="line"></span><br><span class="line"> BeanRegistrationUtil.registerBeanDefinitionIfNotExists(registry, PropertySourcesPlaceholderConfigurer.class.getName(),</span><br><span class="line"> PropertySourcesPlaceholderConfigurer.class, propertySourcesPlaceholderPropertyValues);</span><br><span class="line"> <span class="comment">// 注册了这个 bean</span></span><br><span class="line"> BeanRegistrationUtil.registerBeanDefinitionIfNotExists(registry, PropertySourcesProcessor.class.getName(),</span><br><span class="line"> PropertySourcesProcessor.class);</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>而com.ctrip.framework.apollo.spring.config.PropertySourcesProcessor 实现了 org.springframework.beans.factory.config.BeanFactoryPostProcessor<br>它里面的 com.ctrip.framework.apollo.spring.config.PropertySourcesProcessor#postProcessBeanFactory 方法就会被 spring 调用，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">initializePropertySources</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (environment.getPropertySources().contains(PropertySourcesConstants.APOLLO_PROPERTY_SOURCE_NAME)) {</span><br><span class="line"> <span class="comment">//already initialized</span></span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">CompositePropertySource</span> <span class="variable">composite</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">CompositePropertySource</span>(PropertySourcesConstants.APOLLO_PROPERTY_SOURCE_NAME);</span><br><span class="line"></span><br><span class="line"> <span class="comment">//sort by order asc</span></span><br><span class="line"> ImmutableSortedSet<Integer> orders = ImmutableSortedSet.copyOf(NAMESPACE_NAMES.keySet());</span><br><span class="line"> Iterator<Integer> iterator = orders.iterator();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">while</span> (iterator.hasNext()) {</span><br><span class="line"> <span class="type">int</span> <span class="variable">order</span> <span class="operator">=</span> iterator.next();</span><br><span class="line"> <span class="keyword">for</span> (String namespace : NAMESPACE_NAMES.get(order)) {</span><br><span class="line"> <span class="comment">// 这里获取每个 namespace 的配置</span></span><br><span class="line"> <span class="type">Config</span> <span class="variable">config</span> <span class="operator">=</span> ConfigService.getConfig(namespace);</span><br><span class="line"></span><br><span class="line"> composite.addPropertySource(configPropertySourceFactory.getConfigPropertySource(namespace, config));</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后是 com.ctrip.framework.apollo.ConfigService#getConfig<br>接着就是它<br>com.ctrip.framework.apollo.internals.DefaultConfigManager#getConfig</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> Config <span class="title function_">getConfig</span><span class="params">(String namespace)</span> {</span><br><span class="line"> <span class="type">Config</span> <span class="variable">config</span> <span class="operator">=</span> m_configs.get(namespace);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (config == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">synchronized</span> (<span class="built_in">this</span>) {</span><br><span class="line"> config = m_configs.get(namespace);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (config == <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">ConfigFactory</span> <span class="variable">factory</span> <span class="operator">=</span> m_factoryManager.getFactory(namespace);</span><br><span class="line"></span><br><span class="line"><span class="comment">// 通过 factory 来创建配置获取</span></span><br><span class="line"> config = factory.create(namespace);</span><br><span class="line"> m_configs.put(namespace, config);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>创建配置</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">com.ctrip.framework.apollo.spi.DefaultConfigFactory#create</span><br><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> Config <span class="title function_">create</span><span class="params">(String namespace)</span> {</span><br><span class="line"> <span class="type">ConfigFileFormat</span> <span class="variable">format</span> <span class="operator">=</span> determineFileFormat(namespace);</span><br><span class="line"> <span class="keyword">if</span> (ConfigFileFormat.isPropertiesCompatible(format)) {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">DefaultConfig</span>(namespace, createPropertiesCompatibleFileConfigRepository(namespace, format));</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 调用到这</span></span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">DefaultConfig</span>(namespace, createLocalConfigRepository(namespace));</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">LocalFileConfigRepository <span class="title function_">createLocalConfigRepository</span><span class="params">(String namespace)</span> {</span><br><span class="line"> <span class="keyword">if</span> (m_configUtil.isInLocalMode()) {</span><br><span class="line"> logger.warn(</span><br><span class="line"> <span class="string">"==== Apollo is in local mode! Won't pull configs from remote server for namespace {} ! ===="</span>,</span><br><span class="line"> namespace);</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">LocalFileConfigRepository</span>(namespace);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 正常会走这个，因为要从配置中心获取</span></span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">LocalFileConfigRepository</span>(namespace, createRemoteConfigRepository(namespace));</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后是创建远程配置仓库</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">com.ctrip.framework.apollo.spi.DefaultConfigFactory#createRemoteConfigRepository</span><br><span class="line"> RemoteConfigRepository <span class="title function_">createRemoteConfigRepository</span><span class="params">(String namespace)</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">RemoteConfigRepository</span>(namespace);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>继续对当前的 namespace 创建远程配置仓库</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">com.ctrip.framework.apollo.internals.RemoteConfigRepository#RemoteConfigRepository</span><br><span class="line"><span class="keyword">public</span> <span class="title function_">RemoteConfigRepository</span><span class="params">(String namespace)</span> {</span><br><span class="line"> m_namespace = namespace;</span><br><span class="line"> m_configCache = <span class="keyword">new</span> <span class="title class_">AtomicReference</span><>();</span><br><span class="line"> m_configUtil = ApolloInjector.getInstance(ConfigUtil.class);</span><br><span class="line"> m_httpUtil = ApolloInjector.getInstance(HttpUtil.class);</span><br><span class="line"> m_serviceLocator = ApolloInjector.getInstance(ConfigServiceLocator.class);</span><br><span class="line"> remoteConfigLongPollService = ApolloInjector.getInstance(RemoteConfigLongPollService.class);</span><br><span class="line"> m_longPollServiceDto = <span class="keyword">new</span> <span class="title class_">AtomicReference</span><>();</span><br><span class="line"> m_remoteMessages = <span class="keyword">new</span> <span class="title class_">AtomicReference</span><>();</span><br><span class="line"> m_loadConfigRateLimiter = RateLimiter.create(m_configUtil.getLoadConfigQPS());</span><br><span class="line"> m_configNeedForceRefresh = <span class="keyword">new</span> <span class="title class_">AtomicBoolean</span>(<span class="literal">true</span>);</span><br><span class="line"> m_loadConfigFailSchedulePolicy = <span class="keyword">new</span> <span class="title class_">ExponentialSchedulePolicy</span>(m_configUtil.getOnErrorRetryInterval(),</span><br><span class="line"> m_configUtil.getOnErrorRetryInterval() * <span class="number">8</span>);</span><br><span class="line"> gson = <span class="keyword">new</span> <span class="title class_">Gson</span>();</span><br><span class="line"> <span class="comment">// 尝试同步</span></span><br><span class="line"> <span class="built_in">this</span>.trySync();</span><br><span class="line"> <span class="built_in">this</span>.schedulePeriodicRefresh();</span><br><span class="line"> <span class="built_in">this</span>.scheduleLongPollingRefresh();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后是同步配置，下面的日志异常经常可以看到，比如配置拉取地址不通</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">com.ctrip.framework.apollo.internals.AbstractConfigRepository#trySync</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">boolean</span> <span class="title function_">trySync</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> sync();</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable ex) {</span><br><span class="line"> Tracer.logEvent(<span class="string">"ApolloConfigException"</span>, ExceptionUtil.getDetailMessage(ex));</span><br><span class="line"> logger</span><br><span class="line"> .warn(<span class="string">"Sync config failed, will retry. Repository {}, reason: {}"</span>, <span class="built_in">this</span>.getClass(), ExceptionUtil</span><br><span class="line"> .getDetailMessage(ex));</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>实际的同步方法，加了<code>synchronized </code>锁，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">com.ctrip.framework.apollo.internals.RemoteConfigRepository#sync</span><br><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title function_">sync</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">Transaction</span> <span class="variable">transaction</span> <span class="operator">=</span> Tracer.newTransaction(<span class="string">"Apollo.ConfigService"</span>, <span class="string">"syncRemoteConfig"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 获取本地配置</span></span><br><span class="line"> <span class="type">ApolloConfig</span> <span class="variable">previous</span> <span class="operator">=</span> m_configCache.get();</span><br><span class="line"> <span class="comment">// 获取配置</span></span><br><span class="line"> <span class="type">ApolloConfig</span> <span class="variable">current</span> <span class="operator">=</span> loadApolloConfig();</span><br><span class="line"></span><br><span class="line"> <span class="comment">//reference equals means HTTP 304</span></span><br><span class="line"> <span class="keyword">if</span> (previous != current) {</span><br><span class="line"> logger.debug(<span class="string">"Remote Config refreshed!"</span>);</span><br><span class="line"> m_configCache.set(current);</span><br><span class="line"> <span class="built_in">this</span>.fireRepositoryChange(m_namespace, <span class="built_in">this</span>.getConfig());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (current != <span class="literal">null</span>) {</span><br><span class="line"> Tracer.logEvent(String.format(<span class="string">"Apollo.Client.Configs.%s"</span>, current.getNamespaceName()),</span><br><span class="line"> current.getReleaseKey());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> transaction.setStatus(Transaction.SUCCESS);</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable ex) {</span><br><span class="line"> transaction.setStatus(ex);</span><br><span class="line"> <span class="keyword">throw</span> ex;</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> transaction.complete();</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后走到这</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">com.ctrip.framework.apollo.internals.RemoteConfigRepository#loadApolloConfig</span><br><span class="line"><span class="keyword">private</span> ApolloConfig <span class="title function_">loadApolloConfig</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (!m_loadConfigRateLimiter.tryAcquire(<span class="number">5</span>, TimeUnit.SECONDS)) {</span><br><span class="line"> <span class="comment">//wait at most 5 seconds</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> TimeUnit.SECONDS.sleep(<span class="number">5</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="type">String</span> <span class="variable">appId</span> <span class="operator">=</span> m_configUtil.getAppId();</span><br><span class="line"> <span class="type">String</span> <span class="variable">cluster</span> <span class="operator">=</span> m_configUtil.getCluster();</span><br><span class="line"> <span class="type">String</span> <span class="variable">dataCenter</span> <span class="operator">=</span> m_configUtil.getDataCenter();</span><br><span class="line"> Tracer.logEvent(<span class="string">"Apollo.Client.ConfigMeta"</span>, STRING_JOINER.join(appId, cluster, m_namespace));</span><br><span class="line"> <span class="type">int</span> <span class="variable">maxRetries</span> <span class="operator">=</span> m_configNeedForceRefresh.get() ? <span class="number">2</span> : <span class="number">1</span>;</span><br><span class="line"> <span class="type">long</span> <span class="variable">onErrorSleepTime</span> <span class="operator">=</span> <span class="number">0</span>; <span class="comment">// 0 means no sleep</span></span><br><span class="line"> <span class="type">Throwable</span> <span class="variable">exception</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 获取配置</span></span><br><span class="line"> List<ServiceDTO> configServices = getConfigServices();</span><br><span class="line"> <span class="type">String</span> <span class="variable">url</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < maxRetries; i++) {</span><br><span class="line"> List<ServiceDTO> randomConfigServices = Lists.newLinkedList(configServices);</span><br><span class="line"> Collections.shuffle(randomConfigServices);</span><br><span class="line"> <span class="comment">//Access the server which notifies the client first</span></span><br><span class="line"> <span class="keyword">if</span> (m_longPollServiceDto.get() != <span class="literal">null</span>) {</span><br><span class="line"> randomConfigServices.add(<span class="number">0</span>, m_longPollServiceDto.getAndSet(<span class="literal">null</span>));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (ServiceDTO configService : randomConfigServices) {</span><br><span class="line"> <span class="keyword">if</span> (onErrorSleepTime > <span class="number">0</span>) {</span><br><span class="line"> logger.warn(</span><br><span class="line"> <span class="string">"Load config failed, will retry in {} {}. appId: {}, cluster: {}, namespaces: {}"</span>,</span><br><span class="line"> onErrorSleepTime, m_configUtil.getOnErrorRetryIntervalTimeUnit(), appId, cluster, m_namespace);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> m_configUtil.getOnErrorRetryIntervalTimeUnit().sleep(onErrorSleepTime);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> <span class="comment">//ignore</span></span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> url = assembleQueryConfigUrl(configService.getHomepageUrl(), appId, cluster, m_namespace,</span><br><span class="line"> dataCenter, m_remoteMessages.get(), m_configCache.get());</span><br><span class="line"></span><br><span class="line"> logger.debug(<span class="string">"Loading config from {}"</span>, url);</span><br><span class="line"> <span class="type">HttpRequest</span> <span class="variable">request</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">HttpRequest</span>(url);</span><br><span class="line"></span><br><span class="line"> <span class="type">Transaction</span> <span class="variable">transaction</span> <span class="operator">=</span> Tracer.newTransaction(<span class="string">"Apollo.ConfigService"</span>, <span class="string">"queryConfig"</span>);</span><br><span class="line"> transaction.addData(<span class="string">"Url"</span>, url);</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"></span><br><span class="line"> HttpResponse<ApolloConfig> response = m_httpUtil.doGet(request, ApolloConfig.class);</span><br><span class="line"> m_configNeedForceRefresh.set(<span class="literal">false</span>);</span><br><span class="line"> m_loadConfigFailSchedulePolicy.success();</span><br><span class="line"></span><br><span class="line"> transaction.addData(<span class="string">"StatusCode"</span>, response.getStatusCode());</span><br><span class="line"> transaction.setStatus(Transaction.SUCCESS);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (response.getStatusCode() == <span class="number">304</span>) {</span><br><span class="line"> logger.debug(<span class="string">"Config server responds with 304 HTTP status code."</span>);</span><br><span class="line"> <span class="keyword">return</span> m_configCache.get();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">ApolloConfig</span> <span class="variable">result</span> <span class="operator">=</span> response.getBody();</span><br><span class="line"></span><br><span class="line"> logger.debug(<span class="string">"Loaded config for {}: {}"</span>, m_namespace, result);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> result;</span><br><span class="line"> } <span class="keyword">catch</span> (ApolloConfigStatusCodeException ex) {</span><br><span class="line"> <span class="type">ApolloConfigStatusCodeException</span> <span class="variable">statusCodeException</span> <span class="operator">=</span> ex;</span><br><span class="line"> <span class="comment">//config not found</span></span><br><span class="line"> <span class="keyword">if</span> (ex.getStatusCode() == <span class="number">404</span>) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">message</span> <span class="operator">=</span> String.format(</span><br><span class="line"> <span class="string">"Could not find config for namespace - appId: %s, cluster: %s, namespace: %s, "</span> +</span><br><span class="line"> <span class="string">"please check whether the configs are released in Apollo!"</span>,</span><br><span class="line"> appId, cluster, m_namespace);</span><br><span class="line"> statusCodeException = <span class="keyword">new</span> <span class="title class_">ApolloConfigStatusCodeException</span>(ex.getStatusCode(),</span><br><span class="line"> message);</span><br><span class="line"> }</span><br><span class="line"> Tracer.logEvent(<span class="string">"ApolloConfigException"</span>, ExceptionUtil.getDetailMessage(statusCodeException));</span><br><span class="line"> transaction.setStatus(statusCodeException);</span><br><span class="line"> exception = statusCodeException;</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable ex) {</span><br><span class="line"> Tracer.logEvent(<span class="string">"ApolloConfigException"</span>, ExceptionUtil.getDetailMessage(ex));</span><br><span class="line"> transaction.setStatus(ex);</span><br><span class="line"> exception = ex;</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> transaction.complete();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// if force refresh, do normal sleep, if normal config load, do exponential sleep</span></span><br><span class="line"> onErrorSleepTime = m_configNeedForceRefresh.get() ? m_configUtil.getOnErrorRetryInterval() :</span><br><span class="line"> m_loadConfigFailSchedulePolicy.fail();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"> <span class="type">String</span> <span class="variable">message</span> <span class="operator">=</span> String.format(</span><br><span class="line"> <span class="string">"Load Apollo Config failed - appId: %s, cluster: %s, namespace: %s, url: %s"</span>,</span><br><span class="line"> appId, cluster, m_namespace, url);</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">ApolloConfigException</span>(message, exception);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">com.ctrip.framework.apollo.internals.RemoteConfigRepository#getConfigServices</span><br><span class="line"> <span class="keyword">private</span> List<ServiceDTO> <span class="title function_">getConfigServices</span><span class="params">()</span> {</span><br><span class="line"> List<ServiceDTO> services = m_serviceLocator.getConfigServices();</span><br><span class="line"> <span class="keyword">if</span> (services.size() == <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">ApolloConfigException</span>(<span class="string">"No available config service"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> services;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">com.ctrip.framework.apollo.internals.ConfigServiceLocator#getConfigServices</span><br><span class="line"> <span class="keyword">public</span> List<ServiceDTO> <span class="title function_">getConfigServices</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (m_configServices.get().isEmpty()) {</span><br><span class="line"> updateConfigServices();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> m_configServices.get();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>更新配置服务</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">com.ctrip.framework.apollo.internals.ConfigServiceLocator#updateConfigServices</span><br><span class="line"><span class="keyword">private</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title function_">updateConfigServices</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">url</span> <span class="operator">=</span> assembleMetaServiceUrl();</span><br><span class="line"></span><br><span class="line"> <span class="type">HttpRequest</span> <span class="variable">request</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">HttpRequest</span>(url);</span><br><span class="line"> <span class="type">int</span> <span class="variable">maxRetries</span> <span class="operator">=</span> <span class="number">2</span>;</span><br><span class="line"> <span class="type">Throwable</span> <span class="variable">exception</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < maxRetries; i++) {</span><br><span class="line"> <span class="type">Transaction</span> <span class="variable">transaction</span> <span class="operator">=</span> Tracer.newTransaction(<span class="string">"Apollo.MetaService"</span>, <span class="string">"getConfigService"</span>);</span><br><span class="line"> transaction.addData(<span class="string">"Url"</span>, url);</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 发起 http 请求获取配置</span></span><br><span class="line"> HttpResponse<List<ServiceDTO>> response = m_httpUtil.doGet(request, m_responseType);</span><br><span class="line"> transaction.setStatus(Transaction.SUCCESS);</span><br><span class="line"> List<ServiceDTO> services = response.getBody();</span><br><span class="line"> <span class="keyword">if</span> (services == <span class="literal">null</span> || services.isEmpty()) {</span><br><span class="line"> logConfigService(<span class="string">"Empty response!"</span>);</span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"> setConfigServices(services);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable ex) {</span><br><span class="line"> Tracer.logEvent(<span class="string">"ApolloConfigException"</span>, ExceptionUtil.getDetailMessage(ex));</span><br><span class="line"> transaction.setStatus(ex);</span><br><span class="line"> exception = ex;</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> transaction.complete();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> m_configUtil.getOnErrorRetryIntervalTimeUnit().sleep(m_configUtil.getOnErrorRetryInterval());</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException ex) {</span><br><span class="line"> <span class="comment">//ignore</span></span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">ApolloConfigException</span>(</span><br><span class="line"> String.format(<span class="string">"Get config services failed from %s"</span>, url), exception);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Apollo</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Apollo 的 value 注解是怎么自动更新的</title>
|
|
<url>/2020/11/01/Apollo-%E7%9A%84-value-%E6%B3%A8%E8%A7%A3%E6%98%AF%E6%80%8E%E4%B9%88%E8%87%AA%E5%8A%A8%E6%9B%B4%E6%96%B0%E7%9A%84/</url>
|
|
<content><![CDATA[<p>在前司和目前公司，用的配置中心都是使用的 Apollo，经过了业界验证，比较强大的配置管理系统，特别是在0.10 后开始支持对使用 value 注解的配置值进行自动更新，今天刚好有个同学问到我，就顺便写篇文章记录下，其实也是借助于 spring 强大的 bean 生命周期管理，可以实现BeanPostProcessor接口，使用postProcessBeforeInitialization方法，来对bean 内部的属性和方法进行判断，是否有 value 注解，如果有就是将它注册到一个 map 中，可以看到这个方法<code>com.ctrip.framework.apollo.spring.annotation.SpringValueProcessor#processField</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">processField</span><span class="params">(Object bean, String beanName, Field field)</span> {</span><br><span class="line"> <span class="comment">// register @Value on field</span></span><br><span class="line"> <span class="type">Value</span> <span class="variable">value</span> <span class="operator">=</span> field.getAnnotation(Value.class);</span><br><span class="line"> <span class="keyword">if</span> (value == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> Set<String> keys = placeholderHelper.extractPlaceholderKeys(value.value());</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (keys.isEmpty()) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (String key : keys) {</span><br><span class="line"> <span class="type">SpringValue</span> <span class="variable">springValue</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SpringValue</span>(key, value.value(), bean, beanName, field, <span class="literal">false</span>);</span><br><span class="line"> springValueRegistry.register(beanFactory, key, springValue);</span><br><span class="line"> logger.debug(<span class="string">"Monitoring {}"</span>, springValue);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后我们看下这个<code>springValueRegistry</code>是啥玩意</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">SpringValueRegistry</span> {</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">long</span> <span class="variable">CLEAN_INTERVAL_IN_SECONDS</span> <span class="operator">=</span> <span class="number">5</span>;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> Map<BeanFactory, Multimap<String, SpringValue>> registry = Maps.newConcurrentMap();</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="type">AtomicBoolean</span> <span class="variable">initialized</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">AtomicBoolean</span>(<span class="literal">false</span>);</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="type">Object</span> <span class="variable">LOCK</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Object</span>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">register</span><span class="params">(BeanFactory beanFactory, String key, SpringValue springValue)</span> {</span><br><span class="line"> <span class="keyword">if</span> (!registry.containsKey(beanFactory)) {</span><br><span class="line"> <span class="keyword">synchronized</span> (LOCK) {</span><br><span class="line"> <span class="keyword">if</span> (!registry.containsKey(beanFactory)) {</span><br><span class="line"> registry.put(beanFactory, LinkedListMultimap.<String, SpringValue>create());</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> registry.get(beanFactory).put(key, springValue);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// lazy initialize</span></span><br><span class="line"> <span class="keyword">if</span> (initialized.compareAndSet(<span class="literal">false</span>, <span class="literal">true</span>)) {</span><br><span class="line"> initialize();</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这类其实就是个 map 来存放 springvalue，然后有<code>com.ctrip.framework.apollo.spring.property.AutoUpdateConfigChangeListener</code>来监听更新操作，当有变更时</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">onChange</span><span class="params">(ConfigChangeEvent changeEvent)</span> {</span><br><span class="line"> Set<String> keys = changeEvent.changedKeys();</span><br><span class="line"> <span class="keyword">if</span> (CollectionUtils.isEmpty(keys)) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">for</span> (String key : keys) {</span><br><span class="line"> <span class="comment">// 1. check whether the changed key is relevant</span></span><br><span class="line"> Collection<SpringValue> targetValues = springValueRegistry.get(beanFactory, key);</span><br><span class="line"> <span class="keyword">if</span> (targetValues == <span class="literal">null</span> || targetValues.isEmpty()) {</span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 2. check whether the value is really changed or not (since spring property sources have hierarchies)</span></span><br><span class="line"> <span class="comment">// 这里其实有一点比较绕，是因为 Apollo 里的 namespace 划分，会出现 key 相同，但是 namespace 不同的情况，所以会有个优先级存在，所以需要去校验 environment 里面的是否已经更新，如果未更新则表示不需要更新</span></span><br><span class="line"> <span class="keyword">if</span> (!shouldTriggerAutoUpdate(changeEvent, key)) {</span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 3. update the value</span></span><br><span class="line"> <span class="keyword">for</span> (SpringValue val : targetValues) {</span><br><span class="line"> updateSpringValue(val);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>其实原理很简单，就是得了解知道下</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Apollo</category>
|
|
<category>value</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Apollo</tag>
|
|
<tag>environment</tag>
|
|
<tag>value</tag>
|
|
<tag>注解</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Clone Graph Part I</title>
|
|
<url>/2014/12/30/Clone-Graph-Part-I/</url>
|
|
<content><![CDATA[<h3 id="problem"><a href="#problem" class="headerlink" title="problem"></a>problem</h3><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Clone a graph. Input is a Node pointer. Return the Node pointer of the cloned graph.</span><br><span class="line"></span><br><span class="line">A graph is defined below:</span><br><span class="line">struct Node {</span><br><span class="line">vector neighbors;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<span id="more"></span>
|
|
<h3 id="code"><a href="#code" class="headerlink" title="code"></a>code</h3><figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">typedef</span> unordered_map<Node *, Node *> Map;</span><br><span class="line"> </span><br><span class="line"><span class="function">Node *<span class="title">clone</span><span class="params">(Node *graph)</span> </span>{</span><br><span class="line"> <span class="keyword">if</span> (!graph) <span class="keyword">return</span> <span class="literal">NULL</span>;</span><br><span class="line"> </span><br><span class="line"> Map map;</span><br><span class="line"> queue<Node *> q;</span><br><span class="line"> q.<span class="built_in">push</span>(graph);</span><br><span class="line"> </span><br><span class="line"> Node *graphCopy = <span class="keyword">new</span> <span class="built_in">Node</span>();</span><br><span class="line"> map[graph] = graphCopy;</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">while</span> (!q.<span class="built_in">empty</span>()) {</span><br><span class="line"> Node *node = q.<span class="built_in">front</span>();</span><br><span class="line"> q.<span class="built_in">pop</span>();</span><br><span class="line"> <span class="type">int</span> n = node->neighbors.<span class="built_in">size</span>();</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> i = <span class="number">0</span>; i < n; i++) {</span><br><span class="line"> Node *neighbor = node->neighbors[i];</span><br><span class="line"> <span class="comment">// no copy exists</span></span><br><span class="line"> <span class="keyword">if</span> (map.<span class="built_in">find</span>(neighbor) == map.<span class="built_in">end</span>()) {</span><br><span class="line"> Node *p = <span class="keyword">new</span> <span class="built_in">Node</span>();</span><br><span class="line"> map[node]->neighbors.<span class="built_in">push_back</span>(p);</span><br><span class="line"> map[neighbor] = p;</span><br><span class="line"> q.<span class="built_in">push</span>(neighbor);</span><br><span class="line"> } <span class="keyword">else</span> { <span class="comment">// a copy already exists</span></span><br><span class="line"> map[node]->neighbors.<span class="built_in">push_back</span>(map[neighbor]);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">return</span> graphCopy;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<h3 id="anlysis"><a href="#anlysis" class="headerlink" title="anlysis"></a>anlysis</h3><p>using the Breadth-first traversal<br>and use a map to save the neighbors not to be duplicated.</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>C++</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Comparator使用小记</title>
|
|
<url>/2020/04/05/Comparator%E4%BD%BF%E7%94%A8%E5%B0%8F%E8%AE%B0/</url>
|
|
<content><![CDATA[<p>在Java8的stream之前，将对象进行排序的时候，可能需要对象实现Comparable接口，或者自己实现一个Comparator，</p>
|
|
<p>比如这样子</p>
|
|
<p>我的对象是Entity</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">Entity</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> Long id;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> Long sortValue;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> Long <span class="title function_">getId</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> id;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">setId</span><span class="params">(Long id)</span> {</span><br><span class="line"> <span class="built_in">this</span>.id = id;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> Long <span class="title function_">getSortValue</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> sortValue;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">setSortValue</span><span class="params">(Long sortValue)</span> {</span><br><span class="line"> <span class="built_in">this</span>.sortValue = sortValue;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>Comparator</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">MyComparator</span> <span class="keyword">implements</span> <span class="title class_">Comparator</span> {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="type">int</span> <span class="title function_">compare</span><span class="params">(Object o1, Object o2)</span> {</span><br><span class="line"> <span class="type">Entity</span> <span class="variable">e1</span> <span class="operator">=</span> (Entity) o1;</span><br><span class="line"> <span class="type">Entity</span> <span class="variable">e2</span> <span class="operator">=</span> (Entity) o2;</span><br><span class="line"> <span class="keyword">if</span> (e1.getSortValue() < e2.getSortValue()) {</span><br><span class="line"> <span class="keyword">return</span> -<span class="number">1</span>;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (e1.getSortValue().equals(e2.getSortValue())) {</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>比较代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="type">MyComparator</span> <span class="variable">myComparator</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">MyComparator</span>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> List<Entity> list = <span class="keyword">new</span> <span class="title class_">ArrayList</span><Entity>();</span><br><span class="line"> <span class="type">Entity</span> <span class="variable">e1</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Entity</span>();</span><br><span class="line"> e1.setId(<span class="number">1L</span>);</span><br><span class="line"> e1.setSortValue(<span class="number">1L</span>);</span><br><span class="line"> list.add(e1);</span><br><span class="line"> <span class="type">Entity</span> <span class="variable">e2</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Entity</span>();</span><br><span class="line"> e2.setId(<span class="number">2L</span>);</span><br><span class="line"> e2.setSortValue(<span class="literal">null</span>);</span><br><span class="line"> list.add(e2);</span><br><span class="line"> Collections.sort(list, myComparator);</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>看到这里的e2的排序值是null，在Comparator中如果要正常运行的话，就得判空之类的，这里有两点需要，一个是不想写这个MyComparator，然后也没那么好排除掉list里排序值，那么有什么办法能解决这种问题呢，应该说java的这方面真的是很强大</p>
|
|
<p><img data-src="https://i.loli.net/2020/04/05/8VfjeOSmcvx4dyk.png"></p>
|
|
<p>看一下nullsFirst的实现</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">final</span> <span class="keyword">static</span> <span class="keyword">class</span> <span class="title class_">NullComparator</span><T> <span class="keyword">implements</span> <span class="title class_">Comparator</span><T>, Serializable {</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">long</span> <span class="variable">serialVersionUID</span> <span class="operator">=</span> -<span class="number">7569533591570686392L</span>;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="type">boolean</span> nullFirst;</span><br><span class="line"> <span class="comment">// if null, non-null Ts are considered equal</span></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> Comparator<T> real;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@SuppressWarnings("unchecked")</span></span><br><span class="line"> NullComparator(<span class="type">boolean</span> nullFirst, Comparator<? <span class="built_in">super</span> T> real) {</span><br><span class="line"> <span class="built_in">this</span>.nullFirst = nullFirst;</span><br><span class="line"> <span class="built_in">this</span>.real = (Comparator<T>) real;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="type">int</span> <span class="title function_">compare</span><span class="params">(T a, T b)</span> {</span><br><span class="line"> <span class="keyword">if</span> (a == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> (b == <span class="literal">null</span>) ? <span class="number">0</span> : (nullFirst ? -<span class="number">1</span> : <span class="number">1</span>);</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (b == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> nullFirst ? <span class="number">1</span>: -<span class="number">1</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> (real == <span class="literal">null</span>) ? <span class="number">0</span> : real.compare(a, b);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>核心代码就是下面这段，其实就是帮我们把前面要做的事情做掉了，是不是挺方便的，小记一下哈</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>集合</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Stream</tag>
|
|
<tag>Comparator</tag>
|
|
<tag>排序</tag>
|
|
<tag>sort</tag>
|
|
<tag>nullsfirst</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Disruptor 系列二</title>
|
|
<url>/2022/02/27/Disruptor-%E7%B3%BB%E5%88%97%E4%BA%8C/</url>
|
|
<content><![CDATA[<p>这里开始慢慢深入的讲一下 disruptor，首先是 <code>lock free</code> , 相比于前面介绍的两个阻塞队列，<br>disruptor 本身是不直接使用锁的，因为本身的设计是单个线程去生产，通过 cas 来维护头指针，<br>不直接维护尾指针，这样就减少了锁的使用，提升了性能；第二个是这次介绍的重点，<br>减少 <code>false sharing</code> 的情况，也就是常说的 <em><strong>伪共享</strong></em> 问题，那么什么叫 <em><strong>伪共享</strong></em> 呢，<br>这里要扯到一些 cpu 缓存的知识，<br><img data-src="https://img.nicksxs.me/uPic/2BYYAw.png"><br>譬如我在用的这个笔记本<br><img data-src="https://img.nicksxs.me/uPic/yzWhoW.png"><br>这里就可能看到 L2 Cache 就是针对每个核的<br><img data-src="https://img.nicksxs.me/uPic/q3n0hd.png"><br>这里可以看到现代 CPU 的结构里，分为三级缓存，越靠近 cpu 的速度越快，存储容量越小，<br>而 L1 跟 L2 是 CPU 核专属的每个核都有自己的 L1 和 L2 的，其中 L1 还分为数据和指令，<br>像我上面的图中显示的 L1 Cache 只有 64KB 大小，其中数据 32KB，指令 32KB，<br>而 L2 则有 256KB，L3 有 4MB，其中的 Line Size 是我们这里比较重要的一个值，<br>CPU 其实会就近地从 Cache 中读取数据，碰到 <code>Cache Miss</code> 就再往下一级 Cache 读取，<br>每次读取是按照缓存行 <code>Cache Line</code> 读取，并且也遵循了“就近原则”，<br>也就是相近的数据有可能也会马上被读取，所以以行的形式读取，然而这也造成了 <code>false sharing</code>，<br>因为类似于 <code>ArrayBlockingQueue</code>，需要有 <code>takeIndex</code> , <code>putIndex</code> , <code>count</code> , 因为在同一个类中，<br>很有可能存在于同一个 <code>Cache Line</code> 中，但是这几个值会被不同的线程修改，<br>导致从 Cache 取出来以后立马就会被失效，所谓的就近原则也就没用了，<br>因为需要反复地标记 dirty 脏位，然后把 Cache 刷掉，就造成了<code>false sharing</code>这种情况<br>而在 <code>disruptor</code> 中则使用了填充的方式，让我的头指针能够不产生<code>false sharing</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">LhsPadding</span></span><br><span class="line">{</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">long</span> p1, p2, p3, p4, p5, p6, p7;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">class</span> <span class="title class_">Value</span> <span class="keyword">extends</span> <span class="title class_">LhsPadding</span></span><br><span class="line">{</span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">volatile</span> <span class="type">long</span> value;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">class</span> <span class="title class_">RhsPadding</span> <span class="keyword">extends</span> <span class="title class_">Value</span></span><br><span class="line">{</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">long</span> p9, p10, p11, p12, p13, p14, p15;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * <p>Concurrent sequence class used for tracking the progress of</span></span><br><span class="line"><span class="comment"> * the ring buffer and event processors. Support a number</span></span><br><span class="line"><span class="comment"> * of concurrent operations including CAS and order writes.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p>Also attempts to be more efficient with regards to false</span></span><br><span class="line"><span class="comment"> * sharing by adding padding around the volatile field.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">Sequence</span> <span class="keyword">extends</span> <span class="title class_">RhsPadding</span></span><br><span class="line">{</span><br></pre></td></tr></table></figure>
|
|
<p>通过代码可以看到，sequence 中其实真正有意义的是 value 字段，因为需要在多线程环境下可见也<br>使用了<code>volatile</code> 关键字，而 <code>LhsPadding</code> 和 <code>RhsPadding</code> 分别在value 前后填充了各<br>7 个 <code>long</code> 型的变量，<code>long</code> 型的变量在 Java 中是占用 8 bytes，这样就相当于不管怎么样，<br>value 都会单独使用一个缓存行，使得其不会产生 <code>false sharing</code> 的问题。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Disruptor</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Disruptor 系列三</title>
|
|
<url>/2022/09/25/Disruptor-%E7%B3%BB%E5%88%97%E4%B8%89/</url>
|
|
<content><![CDATA[<p>原来一直有点被误导，<br>gatingSequences用来标识每个 processer 的操作位点，但是怎么记录更新有点搞不清楚<br>其实问题在于 gatingSequences 是个 Sequence 数组，首先要看下怎么加进去的，<br>可以看到是在 <code>com.lmax.disruptor.RingBuffer#addGatingSequences</code> 这个方法里添加<br>首先是 <code>com.lmax.disruptor.dsl.Disruptor#handleEventsWith(com.lmax.disruptor.EventHandler<? super T>...)</code><br>然后执行 <code>com.lmax.disruptor.dsl.Disruptor#createEventProcessors(com.lmax.disruptor.Sequence[], com.lmax.disruptor.EventHandler<? super T>[])</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">EventHandlerGroup<T> <span class="title function_">createEventProcessors</span><span class="params">(</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> Sequence[] barrierSequences,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> EventHandler<? <span class="built_in">super</span> T>[] eventHandlers)</span></span><br><span class="line"> {</span><br><span class="line"> checkNotStarted();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">final</span> Sequence[] processorSequences = <span class="keyword">new</span> <span class="title class_">Sequence</span>[eventHandlers.length];</span><br><span class="line"> <span class="keyword">final</span> <span class="type">SequenceBarrier</span> <span class="variable">barrier</span> <span class="operator">=</span> ringBuffer.newBarrier(barrierSequences);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>, eventHandlersLength = eventHandlers.length; i < eventHandlersLength; i++)</span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">final</span> EventHandler<? <span class="built_in">super</span> T> eventHandler = eventHandlers[i];</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 这里将 handler 包装成一个 BatchEventProcessor</span></span><br><span class="line"> <span class="keyword">final</span> BatchEventProcessor<T> batchEventProcessor =</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">BatchEventProcessor</span><>(ringBuffer, barrier, eventHandler);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (exceptionHandler != <span class="literal">null</span>)</span><br><span class="line"> {</span><br><span class="line"> batchEventProcessor.setExceptionHandler(exceptionHandler);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> consumerRepository.add(batchEventProcessor, eventHandler, barrier);</span><br><span class="line"> processorSequences[i] = batchEventProcessor.getSequence();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> updateGatingSequencesForNextInChain(barrierSequences, processorSequences);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">EventHandlerGroup</span><>(<span class="built_in">this</span>, consumerRepository, processorSequences);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>在 <code>BatchEventProcessor</code> 在类内有个定义 sequence</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">final</span> <span class="type">Sequence</span> <span class="variable">sequence</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Sequence</span>(Sequencer.INITIAL_CURSOR_VALUE);</span><br></pre></td></tr></table></figure>
|
|
<p>然后在上面循环中的这一句取出来</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">processorSequences[i] = batchEventProcessor.getSequence();</span><br></pre></td></tr></table></figure>
|
|
<p>调用<code>com.lmax.disruptor.dsl.Disruptor#updateGatingSequencesForNextInChain</code> 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">updateGatingSequencesForNextInChain</span><span class="params">(<span class="keyword">final</span> Sequence[] barrierSequences, <span class="keyword">final</span> Sequence[] processorSequences)</span></span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">if</span> (processorSequences.length > <span class="number">0</span>)</span><br><span class="line"> {</span><br><span class="line"> <span class="comment">// 然后在这里添加</span></span><br><span class="line"> ringBuffer.addGatingSequences(processorSequences);</span><br><span class="line"> <span class="keyword">for</span> (<span class="keyword">final</span> Sequence barrierSequence : barrierSequences)</span><br><span class="line"> {</span><br><span class="line"> ringBuffer.removeGatingSequence(barrierSequence);</span><br><span class="line"> }</span><br><span class="line"> consumerRepository.unMarkEventProcessorsAsEndOfChain(barrierSequences);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>而如何更新则是在处理器 <code>com.lmax.disruptor.BatchEventProcessor#run</code> 中</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span></span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">if</span> (running.compareAndSet(IDLE, RUNNING))</span><br><span class="line"> {</span><br><span class="line"> sequenceBarrier.clearAlert();</span><br><span class="line"></span><br><span class="line"> notifyStart();</span><br><span class="line"> <span class="keyword">try</span></span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">if</span> (running.get() == RUNNING)</span><br><span class="line"> {</span><br><span class="line"> processEvents();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">finally</span></span><br><span class="line"> {</span><br><span class="line"> notifyShutdown();</span><br><span class="line"> running.set(IDLE);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> {</span><br><span class="line"> <span class="comment">// This is a little bit of guess work. The running state could of changed to HALTED by</span></span><br><span class="line"> <span class="comment">// this point. However, Java does not have compareAndExchange which is the only way</span></span><br><span class="line"> <span class="comment">// to get it exactly correct.</span></span><br><span class="line"> <span class="keyword">if</span> (running.get() == RUNNING)</span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalStateException</span>(<span class="string">"Thread is already running"</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> {</span><br><span class="line"> earlyExit();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后是</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">processEvents</span><span class="params">()</span></span><br><span class="line"> {</span><br><span class="line"> <span class="type">T</span> <span class="variable">event</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="type">long</span> <span class="variable">nextSequence</span> <span class="operator">=</span> sequence.get() + <span class="number">1L</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">while</span> (<span class="literal">true</span>)</span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">try</span></span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">long</span> <span class="variable">availableSequence</span> <span class="operator">=</span> sequenceBarrier.waitFor(nextSequence);</span><br><span class="line"> <span class="keyword">if</span> (batchStartAware != <span class="literal">null</span>)</span><br><span class="line"> {</span><br><span class="line"> batchStartAware.onBatchStart(availableSequence - nextSequence + <span class="number">1</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">while</span> (nextSequence <= availableSequence)</span><br><span class="line"> {</span><br><span class="line"> event = dataProvider.get(nextSequence);</span><br><span class="line"> eventHandler.onEvent(event, nextSequence, nextSequence == availableSequence);</span><br><span class="line"> nextSequence++;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 如果正常处理完，那就是会更新为 availableSequence，因为都处理好了</span></span><br><span class="line"> sequence.set(availableSequence);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">catch</span> (<span class="keyword">final</span> TimeoutException e)</span><br><span class="line"> {</span><br><span class="line"> notifyTimeout(sequence.get());</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">catch</span> (<span class="keyword">final</span> AlertException ex)</span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">if</span> (running.get() != RUNNING)</span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">catch</span> (<span class="keyword">final</span> Throwable ex)</span><br><span class="line"> {</span><br><span class="line"> handleEventException(ex, nextSequence, event);</span><br><span class="line"> <span class="comment">// 如果是异常就只是 nextSequence</span></span><br><span class="line"> sequence.set(nextSequence);</span><br><span class="line"> nextSequence++;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Disruptor</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Dubbo 使用的几个记忆点</title>
|
|
<url>/2022/04/02/Dubbo-%E4%BD%BF%E7%94%A8%E7%9A%84%E5%87%A0%E4%B8%AA%E8%AE%B0%E5%BF%86%E7%82%B9/</url>
|
|
<content><![CDATA[<p>因为后台使用的 dubbo 作为 rpc 框架，并且会有一些日常使用情景有一些小的技巧，在这里做下记录作笔记用</p>
|
|
<h3 id="dubbo-只拉取不注册"><a href="#dubbo-只拉取不注册" class="headerlink" title="dubbo 只拉取不注册"></a>dubbo 只拉取不注册</h3><figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">dubbo:registry</span> <span class="attr">address</span>=<span class="string">"zookeeper://127.0.0.1:2181"</span> <span class="attr">register</span>=<span class="string">"false"</span> /></span></span><br></pre></td></tr></table></figure>
|
|
<p>就是只要 <code>register="false"</code> 就可以了，这样比如我们在开发环境想运行服务，但又不想让开发环境正常的请求调用到本地来，当然这不是唯一的方式，通过 dubbo 2.7 以上的 tag 路由也可以实现或者自行改造拉取和注册服务的逻辑，因为注册到注册中心的其实是一串带参数的 url，还是比较方便改造的。相反的就是只注册，不拉取</p>
|
|
<h3 id="dubbo-只注册不拉取"><a href="#dubbo-只注册不拉取" class="headerlink" title="dubbo 只注册不拉取"></a>dubbo 只注册不拉取</h3><figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">dubbo:registry</span> <span class="attr">address</span>=<span class="string">"zookeeper://127.0.0.1:2181"</span> <span class="attr">subscribe</span>=<span class="string">"false"</span> /></span></span><br></pre></td></tr></table></figure>
|
|
<p>这个使用场景就是如果我这个服务只作为 provider，没有任何调用其他的服务，其实就可以这么设置 </p>
|
|
<h3 id="权重配置"><a href="#权重配置" class="headerlink" title="权重配置"></a>权重配置</h3><figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">dubbo:provider</span> <span class="attr">loadbalance</span>=<span class="string">"random"</span> <span class="attr">weight</span>=<span class="string">"50"</span>/></span></span><br></pre></td></tr></table></figure>
|
|
<p>首先这是在使用了随机的负载均衡策略的时候可以进行配置，并且是对于多个 provider 的情况下，这样其实也可以部分解决上面的只拉取不注册的问题，我把自己的权重调成 0 或者很低</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Dubbo</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Dubbo</tag>
|
|
<tag>RPC</tag>
|
|
<tag>负载均衡</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Filter, Interceptor, Aop, 啥, 啥, 啥? 这些都是啥?</title>
|
|
<url>/2020/08/22/Filter-Intercepter-Aop-%E5%95%A5-%E5%95%A5-%E5%95%A5-%E8%BF%99%E4%BA%9B%E9%83%BD%E6%98%AF%E5%95%A5/</url>
|
|
<content><![CDATA[<p>本来是想取个像现在那些公众号转了又转的文章标题，”面试官再问你xxxxx，就把这篇文章甩给他看”这种标题，但是觉得实在太 low 了，还是用一部我比较喜欢的电影里的一句台词，《人在囧途》里王宝强对着那张老板给他的欠条，看不懂字时候说的那句，这些都是些啥（第四声）<br>当我刚开始面 Java 的时候，其实我真的没注意这方面的东西，实话说就是不知道这些是啥，开发中用过 Interceptor和 Aop，了解 aop 的实现原理，但是不知道 Java web 中的 Filter 是怎么回事，知道 dubbo 的 filter，就这样，所以被问到了的确是回答不出来，可能就觉得这个渣渣，这么简单的都不会，所以还是花点时间来看看这个是个啥，为了避免我口吐芬芳，还是耐下性子来简单说下这几个东西<br>首先是 servlet，怎么去解释这个呢，因为之前是 PHPer，所以比较喜欢用它来举例子，在普通的 PHP 的 web 应用中一般有几部分组成，接受 HTTP 请求的是前置的 nginx 或者 apache，但是这俩玩意都是只能处理静态的请求，远古时代 PHP 和 HTML 混编是通过 apache 的 php module，跟后来 nginx 使用 php-fpm 其实道理类似，就是把请求中需要 PHP 处理的转发给 PHP，在 Java 中呢，是有个比较牛叉的叫 Tomcat 的，它可以把请求转成 servlet，而 servlet 其实就是一种实现了特定接口的 Java 代码，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"></span><br><span class="line"><span class="keyword">package</span> javax.servlet;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> java.io.IOException;</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Defines methods that all servlets must implement.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p></span></span><br><span class="line"><span class="comment"> * A servlet is a small Java program that runs within a Web server. Servlets</span></span><br><span class="line"><span class="comment"> * receive and respond to requests from Web clients, usually across HTTP, the</span></span><br><span class="line"><span class="comment"> * HyperText Transfer Protocol.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p></span></span><br><span class="line"><span class="comment"> * To implement this interface, you can write a generic servlet that extends</span></span><br><span class="line"><span class="comment"> * <code>javax.servlet.GenericServlet</code> or an HTTP servlet that extends</span></span><br><span class="line"><span class="comment"> * <code>javax.servlet.http.HttpServlet</code>.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p></span></span><br><span class="line"><span class="comment"> * This interface defines methods to initialize a servlet, to service requests,</span></span><br><span class="line"><span class="comment"> * and to remove a servlet from the server. These are known as life-cycle</span></span><br><span class="line"><span class="comment"> * methods and are called in the following sequence:</span></span><br><span class="line"><span class="comment"> * <ol></span></span><br><span class="line"><span class="comment"> * <li>The servlet is constructed, then initialized with the <code>init</code></span></span><br><span class="line"><span class="comment"> * method.</span></span><br><span class="line"><span class="comment"> * <li>Any calls from clients to the <code>service</code> method are handled.</span></span><br><span class="line"><span class="comment"> * <li>The servlet is taken out of service, then destroyed with the</span></span><br><span class="line"><span class="comment"> * <code>destroy</code> method, then garbage collected and finalized.</span></span><br><span class="line"><span class="comment"> * </ol></span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p></span></span><br><span class="line"><span class="comment"> * In addition to the life-cycle methods, this interface provides the</span></span><br><span class="line"><span class="comment"> * <code>getServletConfig</code> method, which the servlet can use to get any</span></span><br><span class="line"><span class="comment"> * startup information, and the <code>getServletInfo</code> method, which allows</span></span><br><span class="line"><span class="comment"> * the servlet to return basic information about itself, such as author,</span></span><br><span class="line"><span class="comment"> * version, and copyright.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> GenericServlet</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> javax.servlet.http.HttpServlet</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">interface</span> <span class="title class_">Servlet</span> {</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Called by the servlet container to indicate to a servlet that the servlet</span></span><br><span class="line"><span class="comment"> * is being placed into service.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p></span></span><br><span class="line"><span class="comment"> * The servlet container calls the <code>init</code> method exactly once</span></span><br><span class="line"><span class="comment"> * after instantiating the servlet. The <code>init</code> method must</span></span><br><span class="line"><span class="comment"> * complete successfully before the servlet can receive any requests.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p></span></span><br><span class="line"><span class="comment"> * The servlet container cannot place the servlet into service if the</span></span><br><span class="line"><span class="comment"> * <code>init</code> method</span></span><br><span class="line"><span class="comment"> * <ol></span></span><br><span class="line"><span class="comment"> * <li>Throws a <code>ServletException</code></span></span><br><span class="line"><span class="comment"> * <li>Does not return within a time period defined by the Web server</span></span><br><span class="line"><span class="comment"> * </ol></span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> config</span></span><br><span class="line"><span class="comment"> * a <code>ServletConfig</code> object containing the servlet's</span></span><br><span class="line"><span class="comment"> * configuration and initialization parameters</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@exception</span> ServletException</span></span><br><span class="line"><span class="comment"> * if an exception has occurred that interferes with the</span></span><br><span class="line"><span class="comment"> * servlet's normal operation</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> UnavailableException</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> #getServletConfig</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">init</span><span class="params">(ServletConfig config)</span> <span class="keyword">throws</span> ServletException;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * Returns a {<span class="doctag">@link</span> ServletConfig} object, which contains initialization and</span></span><br><span class="line"><span class="comment"> * startup parameters for this servlet. The <code>ServletConfig</code></span></span><br><span class="line"><span class="comment"> * object returned is the one passed to the <code>init</code> method.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p></span></span><br><span class="line"><span class="comment"> * Implementations of this interface are responsible for storing the</span></span><br><span class="line"><span class="comment"> * <code>ServletConfig</code> object so that this method can return it. The</span></span><br><span class="line"><span class="comment"> * {<span class="doctag">@link</span> GenericServlet} class, which implements this interface, already</span></span><br><span class="line"><span class="comment"> * does this.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> the <code>ServletConfig</code> object that initializes this</span></span><br><span class="line"><span class="comment"> * servlet</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> #init</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> ServletConfig <span class="title function_">getServletConfig</span><span class="params">()</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Called by the servlet container to allow the servlet to respond to a</span></span><br><span class="line"><span class="comment"> * request.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p></span></span><br><span class="line"><span class="comment"> * This method is only called after the servlet's <code>init()</code> method</span></span><br><span class="line"><span class="comment"> * has completed successfully.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p></span></span><br><span class="line"><span class="comment"> * The status code of the response always should be set for a servlet that</span></span><br><span class="line"><span class="comment"> * throws or sends an error.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p></span></span><br><span class="line"><span class="comment"> * Servlets typically run inside multithreaded servlet containers that can</span></span><br><span class="line"><span class="comment"> * handle multiple requests concurrently. Developers must be aware to</span></span><br><span class="line"><span class="comment"> * synchronize access to any shared resources such as files, network</span></span><br><span class="line"><span class="comment"> * connections, and as well as the servlet's class and instance variables.</span></span><br><span class="line"><span class="comment"> * More information on multithreaded programming in Java is available in <a</span></span><br><span class="line"><span class="comment"> * href</span></span><br><span class="line"><span class="comment"> * ="http://java.sun.com/Series/Tutorial/java/threads/multithreaded.html"></span></span><br><span class="line"><span class="comment"> * the Java tutorial on multi-threaded programming</a>.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> req</span></span><br><span class="line"><span class="comment"> * the <code>ServletRequest</code> object that contains the</span></span><br><span class="line"><span class="comment"> * client's request</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> res</span></span><br><span class="line"><span class="comment"> * the <code>ServletResponse</code> object that contains the</span></span><br><span class="line"><span class="comment"> * servlet's response</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@exception</span> ServletException</span></span><br><span class="line"><span class="comment"> * if an exception occurs that interferes with the servlet's</span></span><br><span class="line"><span class="comment"> * normal operation</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@exception</span> IOException</span></span><br><span class="line"><span class="comment"> * if an input or output exception occurs</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">service</span><span class="params">(ServletRequest req, ServletResponse res)</span></span><br><span class="line"> <span class="keyword">throws</span> ServletException, IOException;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Returns information about the servlet, such as author, version, and</span></span><br><span class="line"><span class="comment"> * copyright.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p></span></span><br><span class="line"><span class="comment"> * The string that this method returns should be plain text and not markup</span></span><br><span class="line"><span class="comment"> * of any kind (such as HTML, XML, etc.).</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> a <code>String</code> containing servlet information</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> String <span class="title function_">getServletInfo</span><span class="params">()</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Called by the servlet container to indicate to a servlet that the servlet</span></span><br><span class="line"><span class="comment"> * is being taken out of service. This method is only called once all</span></span><br><span class="line"><span class="comment"> * threads within the servlet's <code>service</code> method have exited or</span></span><br><span class="line"><span class="comment"> * after a timeout period has passed. After the servlet container calls this</span></span><br><span class="line"><span class="comment"> * method, it will not call the <code>service</code> method again on this</span></span><br><span class="line"><span class="comment"> * servlet.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p></span></span><br><span class="line"><span class="comment"> * This method gives the servlet an opportunity to clean up any resources</span></span><br><span class="line"><span class="comment"> * that are being held (for example, memory, file handles, threads) and make</span></span><br><span class="line"><span class="comment"> * sure that any persistent state is synchronized with the servlet's current</span></span><br><span class="line"><span class="comment"> * state in memory.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">destroy</span><span class="params">()</span>;</span><br><span class="line">}</span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
<p>重点看 servlet 的 service方法，就是接受请求，处理完了给响应，不说细节，不然光 Tomcat 的能说半年，所以呢再进一步去理解，其实就能知道，就是一个先后的问题，盗个图<br><img data-src="https://img.nicksxs.me/uPic/2451842-a95c4ece9b4d3833.png"><br>filter 跟后两者最大的不一样其实是一个基于 servlet，在非常外层做的处理，然后是 interceptor 的 prehandle 跟 posthandle，接着才是我们常规的 aop，就这么点事情，做个小试验吧(还是先补段代码吧)</p>
|
|
<h2 id="Filter"><a href="#Filter" class="headerlink" title="Filter"></a>Filter</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// ---------------------------------------------------- FilterChain Methods</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Invoke the next filter in this chain, passing the specified request</span></span><br><span class="line"><span class="comment"> * and response. If there are no more filters in this chain, invoke</span></span><br><span class="line"><span class="comment"> * the <code>service()</code> method of the servlet itself.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> request The servlet request we are processing</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> response The servlet response we are creating</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@exception</span> IOException if an input/output error occurs</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@exception</span> ServletException if a servlet exception occurs</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">doFilter</span><span class="params">(ServletRequest request, ServletResponse response)</span></span><br><span class="line"> <span class="keyword">throws</span> IOException, ServletException {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span>( Globals.IS_SECURITY_ENABLED ) {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">ServletRequest</span> <span class="variable">req</span> <span class="operator">=</span> request;</span><br><span class="line"> <span class="keyword">final</span> <span class="type">ServletResponse</span> <span class="variable">res</span> <span class="operator">=</span> response;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> java.security.AccessController.doPrivileged(</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">java</span>.security.PrivilegedExceptionAction<Void>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> Void <span class="title function_">run</span><span class="params">()</span></span><br><span class="line"> <span class="keyword">throws</span> ServletException, IOException {</span><br><span class="line"> internalDoFilter(req,res);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> );</span><br><span class="line"> } <span class="keyword">catch</span>( PrivilegedActionException pe) {</span><br><span class="line"> <span class="type">Exception</span> <span class="variable">e</span> <span class="operator">=</span> pe.getException();</span><br><span class="line"> <span class="keyword">if</span> (e <span class="keyword">instanceof</span> ServletException)</span><br><span class="line"> <span class="keyword">throw</span> (ServletException) e;</span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span> (e <span class="keyword">instanceof</span> IOException)</span><br><span class="line"> <span class="keyword">throw</span> (IOException) e;</span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span> (e <span class="keyword">instanceof</span> RuntimeException)</span><br><span class="line"> <span class="keyword">throw</span> (RuntimeException) e;</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">ServletException</span>(e.getMessage(), e);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> internalDoFilter(request,response);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">internalDoFilter</span><span class="params">(ServletRequest request,</span></span><br><span class="line"><span class="params"> ServletResponse response)</span></span><br><span class="line"> <span class="keyword">throws</span> IOException, ServletException {</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Call the next filter if there is one</span></span><br><span class="line"> <span class="keyword">if</span> (pos < n) {</span><br><span class="line"> <span class="type">ApplicationFilterConfig</span> <span class="variable">filterConfig</span> <span class="operator">=</span> filters[pos++];</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">Filter</span> <span class="variable">filter</span> <span class="operator">=</span> filterConfig.getFilter();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (request.isAsyncSupported() && <span class="string">"false"</span>.equalsIgnoreCase(</span><br><span class="line"> filterConfig.getFilterDef().getAsyncSupported())) {</span><br><span class="line"> request.setAttribute(Globals.ASYNC_SUPPORTED_ATTR, Boolean.FALSE);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span>( Globals.IS_SECURITY_ENABLED ) {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">ServletRequest</span> <span class="variable">req</span> <span class="operator">=</span> request;</span><br><span class="line"> <span class="keyword">final</span> <span class="type">ServletResponse</span> <span class="variable">res</span> <span class="operator">=</span> response;</span><br><span class="line"> <span class="type">Principal</span> <span class="variable">principal</span> <span class="operator">=</span></span><br><span class="line"> ((HttpServletRequest) req).getUserPrincipal();</span><br><span class="line"></span><br><span class="line"> Object[] args = <span class="keyword">new</span> <span class="title class_">Object</span>[]{req, res, <span class="built_in">this</span>};</span><br><span class="line"> SecurityUtil.doAsPrivilege (<span class="string">"doFilter"</span>, filter, classType, args, principal);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> filter.doFilter(request, response, <span class="built_in">this</span>);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (IOException | ServletException | RuntimeException e) {</span><br><span class="line"> <span class="keyword">throw</span> e;</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> e = ExceptionUtils.unwrapInvocationTargetException(e);</span><br><span class="line"> ExceptionUtils.handleThrowable(e);</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">ServletException</span>(sm.getString(<span class="string">"filterChain.filter"</span>), e);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// We fell off the end of the chain -- call the servlet instance</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (ApplicationDispatcher.WRAP_SAME_OBJECT) {</span><br><span class="line"> lastServicedRequest.set(request);</span><br><span class="line"> lastServicedResponse.set(response);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (request.isAsyncSupported() && !servletSupportsAsync) {</span><br><span class="line"> request.setAttribute(Globals.ASYNC_SUPPORTED_ATTR,</span><br><span class="line"> Boolean.FALSE);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// Use potentially wrapped request from this point</span></span><br><span class="line"> <span class="keyword">if</span> ((request <span class="keyword">instanceof</span> HttpServletRequest) &&</span><br><span class="line"> (response <span class="keyword">instanceof</span> HttpServletResponse) &&</span><br><span class="line"> Globals.IS_SECURITY_ENABLED ) {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">ServletRequest</span> <span class="variable">req</span> <span class="operator">=</span> request;</span><br><span class="line"> <span class="keyword">final</span> <span class="type">ServletResponse</span> <span class="variable">res</span> <span class="operator">=</span> response;</span><br><span class="line"> <span class="type">Principal</span> <span class="variable">principal</span> <span class="operator">=</span></span><br><span class="line"> ((HttpServletRequest) req).getUserPrincipal();</span><br><span class="line"> Object[] args = <span class="keyword">new</span> <span class="title class_">Object</span>[]{req, res};</span><br><span class="line"> SecurityUtil.doAsPrivilege(<span class="string">"service"</span>,</span><br><span class="line"> servlet,</span><br><span class="line"> classTypeUsedInService,</span><br><span class="line"> args,</span><br><span class="line"> principal);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> servlet.service(request, response);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (IOException | ServletException | RuntimeException e) {</span><br><span class="line"> <span class="keyword">throw</span> e;</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> e = ExceptionUtils.unwrapInvocationTargetException(e);</span><br><span class="line"> ExceptionUtils.handleThrowable(e);</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">ServletException</span>(sm.getString(<span class="string">"filterChain.servlet"</span>), e);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="keyword">if</span> (ApplicationDispatcher.WRAP_SAME_OBJECT) {</span><br><span class="line"> lastServicedRequest.set(<span class="literal">null</span>);</span><br><span class="line"> lastServicedResponse.set(<span class="literal">null</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>注意看这一行<br><code>filter.doFilter(request, response, this);</code><br>是不是看懂了，就是个 filter 链，但是这个代码在哪呢，<code>org.apache.catalina.core.ApplicationFilterChain#doFilter</code><br>然后是interceptor，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">doDispatch</span><span class="params">(HttpServletRequest request, HttpServletResponse response)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="type">HttpServletRequest</span> <span class="variable">processedRequest</span> <span class="operator">=</span> request;</span><br><span class="line"> <span class="type">HandlerExecutionChain</span> <span class="variable">mappedHandler</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">multipartRequestParsed</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="type">WebAsyncManager</span> <span class="variable">asyncManager</span> <span class="operator">=</span> WebAsyncUtils.getAsyncManager(request);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">ModelAndView</span> <span class="variable">mv</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="type">Object</span> <span class="variable">dispatchException</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> processedRequest = <span class="built_in">this</span>.checkMultipart(request);</span><br><span class="line"> multipartRequestParsed = processedRequest != request;</span><br><span class="line"> mappedHandler = <span class="built_in">this</span>.getHandler(processedRequest);</span><br><span class="line"> <span class="keyword">if</span> (mappedHandler == <span class="literal">null</span>) {</span><br><span class="line"> <span class="built_in">this</span>.noHandlerFound(processedRequest, response);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">HandlerAdapter</span> <span class="variable">ha</span> <span class="operator">=</span> <span class="built_in">this</span>.getHandlerAdapter(mappedHandler.getHandler());</span><br><span class="line"> <span class="type">String</span> <span class="variable">method</span> <span class="operator">=</span> request.getMethod();</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">isGet</span> <span class="operator">=</span> <span class="string">"GET"</span>.equals(method);</span><br><span class="line"> <span class="keyword">if</span> (isGet || <span class="string">"HEAD"</span>.equals(method)) {</span><br><span class="line"> <span class="type">long</span> <span class="variable">lastModified</span> <span class="operator">=</span> ha.getLastModified(request, mappedHandler.getHandler());</span><br><span class="line"> <span class="keyword">if</span> ((<span class="keyword">new</span> <span class="title class_">ServletWebRequest</span>(request, response)).checkNotModified(lastModified) && isGet) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/** </span></span><br><span class="line"><span class="comment"> * 看这里看这里‼️</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">if</span> (!mappedHandler.applyPreHandle(processedRequest, response)) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> mv = ha.handle(processedRequest, response, mappedHandler.getHandler());</span><br><span class="line"> <span class="keyword">if</span> (asyncManager.isConcurrentHandlingStarted()) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.applyDefaultViewName(processedRequest, mv);</span><br><span class="line"> <span class="comment">/** </span></span><br><span class="line"><span class="comment"> * 再看这里看这里‼️</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> mappedHandler.applyPostHandle(processedRequest, response, mv);</span><br><span class="line"> } <span class="keyword">catch</span> (Exception var20) {</span><br><span class="line"> dispatchException = var20;</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable var21) {</span><br><span class="line"> dispatchException = <span class="keyword">new</span> <span class="title class_">NestedServletException</span>(<span class="string">"Handler dispatch failed"</span>, var21);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.processDispatchResult(processedRequest, response, mappedHandler, mv, (Exception)dispatchException);</span><br><span class="line"> } <span class="keyword">catch</span> (Exception var22) {</span><br><span class="line"> <span class="built_in">this</span>.triggerAfterCompletion(processedRequest, response, mappedHandler, var22);</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable var23) {</span><br><span class="line"> <span class="built_in">this</span>.triggerAfterCompletion(processedRequest, response, mappedHandler, <span class="keyword">new</span> <span class="title class_">NestedServletException</span>(<span class="string">"Handler processing failed"</span>, var23));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="keyword">if</span> (asyncManager.isConcurrentHandlingStarted()) {</span><br><span class="line"> <span class="keyword">if</span> (mappedHandler != <span class="literal">null</span>) {</span><br><span class="line"> mappedHandler.applyAfterConcurrentHandlingStarted(processedRequest, response);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (multipartRequestParsed) {</span><br><span class="line"> <span class="built_in">this</span>.cleanupMultipart(processedRequest);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>代码在哪呢，<code>org.springframework.web.servlet.DispatcherServlet#doDispatch</code>，然后才是我们自己写的 aop，是不是差不多明白了，嗯，接下来是例子<br>写个 filter</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">DemoFilter</span> <span class="keyword">extends</span> <span class="title class_">HttpServlet</span> <span class="keyword">implements</span> <span class="title class_">Filter</span> {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">init</span><span class="params">(FilterConfig filterConfig)</span> <span class="keyword">throws</span> ServletException {</span><br><span class="line"> System.out.println(<span class="string">"==>DemoFilter启动"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">doFilter</span><span class="params">(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain)</span> <span class="keyword">throws</span> IOException, ServletException {</span><br><span class="line"> <span class="comment">// 将请求转换成HttpServletRequest 请求</span></span><br><span class="line"> <span class="type">HttpServletRequest</span> <span class="variable">req</span> <span class="operator">=</span> (HttpServletRequest) servletRequest;</span><br><span class="line"> <span class="type">HttpServletResponse</span> <span class="variable">resp</span> <span class="operator">=</span> (HttpServletResponse) servletResponse;</span><br><span class="line"> System.out.println(<span class="string">"before filter"</span>);</span><br><span class="line"> filterChain.doFilter(req, resp);</span><br><span class="line"> System.out.println(<span class="string">"after filter"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">destroy</span><span class="params">()</span> {</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>因为用的springboot，所以就不写 web.xml 了，写个配置类</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Configuration</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">FilterConfiguration</span> {</span><br><span class="line"> <span class="meta">@Bean</span></span><br><span class="line"> <span class="keyword">public</span> FilterRegistrationBean <span class="title function_">filterDemo4Registration</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">FilterRegistrationBean</span> <span class="variable">registration</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">FilterRegistrationBean</span>();</span><br><span class="line"> <span class="comment">//注入过滤器</span></span><br><span class="line"> registration.setFilter(<span class="keyword">new</span> <span class="title class_">DemoFilter</span>());</span><br><span class="line"> <span class="comment">//拦截规则</span></span><br><span class="line"> registration.addUrlPatterns(<span class="string">"/*"</span>);</span><br><span class="line"> <span class="comment">//过滤器名称</span></span><br><span class="line"> registration.setName(<span class="string">"DemoFilter"</span>);</span><br><span class="line"> <span class="comment">//是否自动注册 false 取消Filter的自动注册</span></span><br><span class="line"> registration.setEnabled(<span class="literal">true</span>);</span><br><span class="line"> <span class="comment">//过滤器顺序</span></span><br><span class="line"> registration.setOrder(<span class="number">1</span>);</span><br><span class="line"> <span class="keyword">return</span> registration;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后再来个 interceptor 和 aop,以及一个简单的请求处理</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">DemoInterceptor</span> <span class="keyword">extends</span> <span class="title class_">HandlerInterceptorAdapter</span> {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">preHandle</span><span class="params">(HttpServletRequest request, HttpServletResponse response, Object handler)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> System.out.println(<span class="string">"preHandle test"</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">postHandle</span><span class="params">(HttpServletRequest request, HttpServletResponse response, Object handler, ModelAndView modelAndView)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> System.out.println(<span class="string">"postHandle test"</span>);</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"><span class="meta">@Aspect</span></span><br><span class="line"><span class="meta">@Component</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">DemoAspect</span> {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Pointcut("execution( public * com.nicksxs.springbootdemo.demo.DemoController.*())")</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">point</span><span class="params">()</span> {</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Before("point()")</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">doBefore</span><span class="params">()</span>{</span><br><span class="line"> System.out.println(<span class="string">"==doBefore=="</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@After("point()")</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">doAfter</span><span class="params">()</span>{</span><br><span class="line"> System.out.println(<span class="string">"==doAfter=="</span>);</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"><span class="meta">@RestController</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">DemoController</span> {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@RequestMapping("/hello")</span></span><br><span class="line"> <span class="meta">@ResponseBody</span></span><br><span class="line"> <span class="keyword">public</span> String <span class="title function_">hello</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="string">"hello world"</span>;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>好了，请求一下，看看 stdout，<br><img data-src="https://img.nicksxs.me/uPic/pT4oXL.png"><br>搞定完事儿~</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Filter</category>
|
|
<category>Interceptor - AOP</category>
|
|
<category>Spring</category>
|
|
<category>Servlet</category>
|
|
<category>Interceptor</category>
|
|
<category>AOP</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Filter</tag>
|
|
<tag>Interceptor</tag>
|
|
<tag>AOP</tag>
|
|
<tag>Spring</tag>
|
|
<tag>Tomcat</tag>
|
|
<tag>Servlet</tag>
|
|
<tag>Web</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>AQS篇一</title>
|
|
<url>/2021/02/14/AQS%E7%AF%87%E4%B8%80/</url>
|
|
<content><![CDATA[<p>很多东西都是时看时新，而且时间长了也会忘，所以再来复习下，也会有一些新的角度看法这次来聊下AQS的内容，主要是这几个点，</p>
|
|
<h2 id="第一个线程"><a href="#第一个线程" class="headerlink" title="第一个线程"></a>第一个线程</h2><p>第一个线程抢到锁了，此时state跟阻塞队列是怎么样的，其实这里是之前没理解对的地方</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Fair version of tryAcquire. Don't grant access unless</span></span><br><span class="line"><span class="comment"> * recursive call or no waiters or is first.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> <span class="type">boolean</span> <span class="title function_">tryAcquire</span><span class="params">(<span class="type">int</span> acquires)</span> {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">Thread</span> <span class="variable">current</span> <span class="operator">=</span> Thread.currentThread();</span><br><span class="line"> <span class="type">int</span> <span class="variable">c</span> <span class="operator">=</span> getState();</span><br><span class="line"> <span class="comment">// 这里如果state还是0说明锁还空着</span></span><br><span class="line"> <span class="keyword">if</span> (c == <span class="number">0</span>) {</span><br><span class="line"> <span class="comment">// 因为是公平锁版本的，先去看下是否阻塞队列里有排着队的</span></span><br><span class="line"> <span class="keyword">if</span> (!hasQueuedPredecessors() &&</span><br><span class="line"> compareAndSetState(<span class="number">0</span>, acquires)) {</span><br><span class="line"> <span class="comment">// 没有排队的，并且state使用cas设置成功的就标记当前占有锁的线程是我</span></span><br><span class="line"> setExclusiveOwnerThread(current);</span><br><span class="line"> <span class="comment">// 然后其实就返回了，包括阻塞队列的head和tail节点和waitStatus都没有设置</span></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span> (current == getExclusiveOwnerThread()) {</span><br><span class="line"> <span class="type">int</span> <span class="variable">nextc</span> <span class="operator">=</span> c + acquires;</span><br><span class="line"> <span class="keyword">if</span> (nextc < <span class="number">0</span>)</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">Error</span>(<span class="string">"Maximum lock count exceeded"</span>);</span><br><span class="line"> setState(nextc);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 这里就是第二个线程会返回false</span></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<h2 id="第二个线程"><a href="#第二个线程" class="headerlink" title="第二个线程"></a>第二个线程</h2><p>当第二个线程进来的时候应该是怎么样，结合代码来看</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Acquires in exclusive mode, ignoring interrupts. Implemented</span></span><br><span class="line"><span class="comment"> * by invoking at least once {<span class="doctag">@link</span> #tryAcquire},</span></span><br><span class="line"><span class="comment"> * returning on success. Otherwise the thread is queued, possibly</span></span><br><span class="line"><span class="comment"> * repeatedly blocking and unblocking, invoking {<span class="doctag">@link</span></span></span><br><span class="line"><span class="comment"> * #tryAcquire} until success. This method can be used</span></span><br><span class="line"><span class="comment"> * to implement method {<span class="doctag">@link</span> Lock#lock}.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> arg the acquire argument. This value is conveyed to</span></span><br><span class="line"><span class="comment"> * {<span class="doctag">@link</span> #tryAcquire} but is otherwise uninterpreted and</span></span><br><span class="line"><span class="comment"> * can represent anything you like.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">final</span> <span class="keyword">void</span> <span class="title function_">acquire</span><span class="params">(<span class="type">int</span> arg)</span> {</span><br><span class="line"> <span class="comment">// 前面第一种情况是tryAcquire直接成功了，这个if判断第一个条件就是false，就不往下执行了</span></span><br><span class="line"> <span class="comment">// 如果是第二个线程，第一个条件获取锁不成功，条件判断!tryAcquire(arg) == true，就会走</span></span><br><span class="line"> <span class="comment">// acquireQueued(addWaiter(Node.EXCLUSIVE), arg)</span></span><br><span class="line"> <span class="keyword">if</span> (!tryAcquire(arg) &&</span><br><span class="line"> acquireQueued(addWaiter(Node.EXCLUSIVE), arg))</span><br><span class="line"> selfInterrupt();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后来看下addWaiter的逻辑</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Creates and enqueues node for current thread and given mode.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> mode Node.EXCLUSIVE for exclusive, Node.SHARED for shared</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> the new node</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">private</span> Node <span class="title function_">addWaiter</span><span class="params">(Node mode)</span> {</span><br><span class="line"> <span class="comment">// 这里是包装成一个node</span></span><br><span class="line"> <span class="type">Node</span> <span class="variable">node</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Node</span>(Thread.currentThread(), mode);</span><br><span class="line"> <span class="comment">// Try the fast path of enq; backup to full enq on failure</span></span><br><span class="line"> <span class="comment">// 最快的方式就是把当前线程的节点放在阻塞队列的最后</span></span><br><span class="line"> <span class="type">Node</span> <span class="variable">pred</span> <span class="operator">=</span> tail;</span><br><span class="line"> <span class="comment">// 只有当tail，也就是pred不为空的时候可以直接接上</span></span><br><span class="line"> <span class="keyword">if</span> (pred != <span class="literal">null</span>) {</span><br><span class="line"> node.prev = pred;</span><br><span class="line"> <span class="comment">// 如果这里cas成功了，就直接接上返回了</span></span><br><span class="line"> <span class="keyword">if</span> (compareAndSetTail(pred, node)) {</span><br><span class="line"> pred.next = node;</span><br><span class="line"> <span class="keyword">return</span> node;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 不然就会继续走到这里</span></span><br><span class="line"> enq(node);</span><br><span class="line"> <span class="keyword">return</span> node;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后就是enq的逻辑了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Inserts node into queue, initializing if necessary. See picture above.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> node the node to insert</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> node's predecessor</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">private</span> Node <span class="title function_">enq</span><span class="params">(<span class="keyword">final</span> Node node)</span> {</span><br><span class="line"> <span class="keyword">for</span> (;;) {</span><br><span class="line"> <span class="comment">// 如果状态没变化的话，tail这时还是null的</span></span><br><span class="line"> <span class="type">Node</span> <span class="variable">t</span> <span class="operator">=</span> tail;</span><br><span class="line"> <span class="keyword">if</span> (t == <span class="literal">null</span>) { <span class="comment">// Must initialize</span></span><br><span class="line"> <span class="comment">// 这里就会初始化头结点，就是个空节点</span></span><br><span class="line"> <span class="keyword">if</span> (compareAndSetHead(<span class="keyword">new</span> <span class="title class_">Node</span>()))</span><br><span class="line"> <span class="comment">// tail也赋值成head</span></span><br><span class="line"> tail = head;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// 这里就设置tail了</span></span><br><span class="line"> node.prev = t;</span><br><span class="line"> <span class="keyword">if</span> (compareAndSetTail(t, node)) {</span><br><span class="line"> t.next = node;</span><br><span class="line"> <span class="keyword">return</span> t;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>所以从这里可以看出来，其实head头结点不是个真实的带有线程的节点，并且不是在第一个线程进来的时候设置的</p>
|
|
<h2 id="解锁"><a href="#解锁" class="headerlink" title="解锁"></a>解锁</h2><p>通过代码来看下</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Attempts to release this lock.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p>If the current thread is the holder of this lock then the hold</span></span><br><span class="line"><span class="comment"> * count is decremented. If the hold count is now zero then the lock</span></span><br><span class="line"><span class="comment"> * is released. If the current thread is not the holder of this</span></span><br><span class="line"><span class="comment"> * lock then {<span class="doctag">@link</span> IllegalMonitorStateException} is thrown.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@throws</span> IllegalMonitorStateException if the current thread does not</span></span><br><span class="line"><span class="comment"> * hold this lock</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">unlock</span><span class="params">()</span> {</span><br><span class="line"> <span class="comment">// 释放锁</span></span><br><span class="line"> sync.release(<span class="number">1</span>);</span><br><span class="line"> }</span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Releases in exclusive mode. Implemented by unblocking one or</span></span><br><span class="line"><span class="comment"> * more threads if {<span class="doctag">@link</span> #tryRelease} returns true.</span></span><br><span class="line"><span class="comment"> * This method can be used to implement method {<span class="doctag">@link</span> Lock#unlock}.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> arg the release argument. This value is conveyed to</span></span><br><span class="line"><span class="comment"> * {<span class="doctag">@link</span> #tryRelease} but is otherwise uninterpreted and</span></span><br><span class="line"><span class="comment"> * can represent anything you like.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> the value returned from {<span class="doctag">@link</span> #tryRelease}</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">final</span> <span class="type">boolean</span> <span class="title function_">release</span><span class="params">(<span class="type">int</span> arg)</span> {</span><br><span class="line"> <span class="comment">// 尝试去释放</span></span><br><span class="line"> <span class="keyword">if</span> (tryRelease(arg)) {</span><br><span class="line"> <span class="type">Node</span> <span class="variable">h</span> <span class="operator">=</span> head;</span><br><span class="line"> <span class="keyword">if</span> (h != <span class="literal">null</span> && h.waitStatus != <span class="number">0</span>)</span><br><span class="line"> unparkSuccessor(h);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"><span class="keyword">protected</span> <span class="keyword">final</span> <span class="type">boolean</span> <span class="title function_">tryRelease</span><span class="params">(<span class="type">int</span> releases)</span> {</span><br><span class="line"> <span class="type">int</span> <span class="variable">c</span> <span class="operator">=</span> getState() - releases;</span><br><span class="line"> <span class="keyword">if</span> (Thread.currentThread() != getExclusiveOwnerThread())</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalMonitorStateException</span>();</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">free</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="comment">// 判断是否完全释放锁，因为可重入</span></span><br><span class="line"> <span class="keyword">if</span> (c == <span class="number">0</span>) {</span><br><span class="line"> free = <span class="literal">true</span>;</span><br><span class="line"> setExclusiveOwnerThread(<span class="literal">null</span>);</span><br><span class="line"> }</span><br><span class="line"> setState(c);</span><br><span class="line"> <span class="keyword">return</span> free;</span><br><span class="line"> }</span><br><span class="line"><span class="comment">// 这段代码和上面的一致，只是为了顺序性，又拷下来看下</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">final</span> <span class="type">boolean</span> <span class="title function_">release</span><span class="params">(<span class="type">int</span> arg)</span> {</span><br><span class="line"> <span class="comment">// 尝试去释放，如果是完全释放，返回的就是true，否则是false</span></span><br><span class="line"> <span class="keyword">if</span> (tryRelease(arg)) {</span><br><span class="line"> <span class="type">Node</span> <span class="variable">h</span> <span class="operator">=</span> head;</span><br><span class="line"> <span class="comment">// 这里判断头结点是否为空以及waitStatus的状态，前面说了head节点其实是</span></span><br><span class="line"> <span class="comment">// 在第二个线程进来的时候初始化的，如果是空的话说明没后续节点，并且waitStatus</span></span><br><span class="line"> <span class="comment">// 也表示了后续的等待状态</span></span><br><span class="line"> <span class="keyword">if</span> (h != <span class="literal">null</span> && h.waitStatus != <span class="number">0</span>)</span><br><span class="line"> unparkSuccessor(h);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Wakes up node's successor, if one exists.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> node the node</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="comment">// 唤醒后继节点</span></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">unparkSuccessor</span><span class="params">(Node node)</span> {</span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> * If status is negative (i.e., possibly needing signal) try</span></span><br><span class="line"><span class="comment"> * to clear in anticipation of signalling. It is OK if this</span></span><br><span class="line"><span class="comment"> * fails or if status is changed by waiting thread.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">ws</span> <span class="operator">=</span> node.waitStatus;</span><br><span class="line"> <span class="keyword">if</span> (ws < <span class="number">0</span>)</span><br><span class="line"> compareAndSetWaitStatus(node, ws, <span class="number">0</span>);</span><br><span class="line"></span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> * Thread to unpark is held in successor, which is normally</span></span><br><span class="line"><span class="comment"> * just the next node. But if cancelled or apparently null,</span></span><br><span class="line"><span class="comment"> * traverse backwards from tail to find the actual</span></span><br><span class="line"><span class="comment"> * non-cancelled successor.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="type">Node</span> <span class="variable">s</span> <span class="operator">=</span> node.next;</span><br><span class="line"> <span class="comment">// 如果后继节点是空或者当前节点取消等待了</span></span><br><span class="line"> <span class="keyword">if</span> (s == <span class="literal">null</span> || s.waitStatus > <span class="number">0</span>) {</span><br><span class="line"> s = <span class="literal">null</span>;</span><br><span class="line"> <span class="comment">// 从后往前找，找到非取消的节点，注意这里不是找到就退出，而是一直找到头</span></span><br><span class="line"> <span class="comment">// 所以不必担心中间有取消的</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">Node</span> <span class="variable">t</span> <span class="operator">=</span> tail; t != <span class="literal">null</span> && t != node; t = t.prev)</span><br><span class="line"> <span class="keyword">if</span> (t.waitStatus <= <span class="number">0</span>)</span><br><span class="line"> s = t;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (s != <span class="literal">null</span>)</span><br><span class="line"> <span class="comment">// 将其唤醒</span></span><br><span class="line"> LockSupport.unpark(s.thread);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
|
|
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>并发</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
<tag>aqs</tag>
|
|
<tag>并发</tag>
|
|
<tag>j.u.c</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Disruptor 系列一</title>
|
|
<url>/2022/02/13/Disruptor-%E7%B3%BB%E5%88%97%E4%B8%80/</url>
|
|
<content><![CDATA[<p>很久之前就听说过这个框架，不过之前有点跟消息队列混起来，这个也是种队列，但不是跟 rocketmq，nsq 那种一样的，而是在进程内部提供队列服务的，偏向于取代<code>ArrayBlockingQueue</code>，因为这个阻塞队列是使用了锁来控制阻塞，关于并发其实有一些通用的最佳实践，就是用锁，即使是 JDK 提供的锁，也是比较耗资源的，当然这是跟不加锁的对比，同样是锁，JDK 的实现还是性能比较优秀的。常见的阻塞队列中例如 <code>ArrayBlockingQueue</code> 和 <code>LinkedBlockingQueue</code> 都有锁的身影的存在，区别在于 <code>ArrayBlockingQueue</code> 是一把锁，后者是两把锁，不过重点不在几把锁，这里其实是两个问题，一个是所谓的 <code>lock free</code>, 对于一个单生产者的 <code>disruptor</code> 来说，因为写入是只有一个线程的，是可以不用加锁，多生产者的时候使用的是 cas 来获取对应的写入坑位，另一个是解决“伪共享”问题，后面可以详细点分析，先介绍下使用<br>首先是数据源</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">LongEvent</span> {</span><br><span class="line"> <span class="keyword">private</span> <span class="type">long</span> value;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">set</span><span class="params">(<span class="type">long</span> value)</span> {</span><br><span class="line"> <span class="built_in">this</span>.value = value;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="type">long</span> <span class="title function_">getValue</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> value;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">setValue</span><span class="params">(<span class="type">long</span> value)</span> {</span><br><span class="line"> <span class="built_in">this</span>.value = value;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>事件生产</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">LongEventFactory</span> <span class="keyword">implements</span> <span class="title class_">EventFactory</span><LongEvent></span><br><span class="line">{</span><br><span class="line"> <span class="keyword">public</span> LongEvent <span class="title function_">newInstance</span><span class="params">()</span></span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">LongEvent</span>();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>事件处理器</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">LongEventHandler</span> <span class="keyword">implements</span> <span class="title class_">EventHandler</span><LongEvent> {</span><br><span class="line"></span><br><span class="line"> <span class="comment">// event 事件，</span></span><br><span class="line"> <span class="comment">// sequence 当前的序列 </span></span><br><span class="line"> <span class="comment">// 是否当前批次最后一个数据</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">onEvent</span><span class="params">(LongEvent event, <span class="type">long</span> sequence, <span class="type">boolean</span> endOfBatch)</span></span><br><span class="line"> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">str</span> <span class="operator">=</span> String.format(<span class="string">"long event : %s l:%s b:%s"</span>, event.getValue(), sequence, endOfBatch);</span><br><span class="line"> System.out.println(str);</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
<p>主方法代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">package</span> disruptor;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> com.lmax.disruptor.RingBuffer;</span><br><span class="line"><span class="keyword">import</span> com.lmax.disruptor.dsl.Disruptor;</span><br><span class="line"><span class="keyword">import</span> com.lmax.disruptor.util.DaemonThreadFactory;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> java.nio.ByteBuffer;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">LongEventMain</span></span><br><span class="line">{</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> <span class="keyword">throws</span> Exception</span><br><span class="line"> {</span><br><span class="line"> <span class="comment">// 这个需要是 2 的幂次，这样在定位的时候只需要位移操作，也能减少各种计算操作</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">bufferSize</span> <span class="operator">=</span> <span class="number">1024</span>; </span><br><span class="line"></span><br><span class="line"> Disruptor<LongEvent> disruptor = </span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">Disruptor</span><>(LongEvent::<span class="keyword">new</span>, bufferSize, DaemonThreadFactory.INSTANCE);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 类似于注册处理器</span></span><br><span class="line"> disruptor.handleEventsWith(<span class="keyword">new</span> <span class="title class_">LongEventHandler</span>());</span><br><span class="line"> <span class="comment">// 或者直接用 lambda</span></span><br><span class="line"> disruptor.handleEventsWith((event, sequence, endOfBatch) -></span><br><span class="line"> System.out.println(<span class="string">"Event: "</span> + event));</span><br><span class="line"> <span class="comment">// 启动我们的 disruptor</span></span><br><span class="line"> disruptor.start(); </span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> RingBuffer<LongEvent> ringBuffer = disruptor.getRingBuffer(); </span><br><span class="line"> <span class="type">ByteBuffer</span> <span class="variable">bb</span> <span class="operator">=</span> ByteBuffer.allocate(<span class="number">8</span>);</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">long</span> <span class="variable">l</span> <span class="operator">=</span> <span class="number">0</span>; <span class="literal">true</span>; l++)</span><br><span class="line"> {</span><br><span class="line"> bb.putLong(<span class="number">0</span>, l);</span><br><span class="line"> <span class="comment">// 生产事件</span></span><br><span class="line"> ringBuffer.publishEvent((event, sequence, buffer) -> event.set(buffer.getLong(<span class="number">0</span>)), bb);</span><br><span class="line"> Thread.sleep(<span class="number">1000</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>运行下可以看到运行结果<br><img data-src="https://img.nicksxs.me/uPic/Esey7l.png"><br>这里其实就只是最简单的使用，生产者只有一个，然后也不是批量的。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Disruptor</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>G1收集器概述</title>
|
|
<url>/2020/02/09/G1%E6%94%B6%E9%9B%86%E5%99%A8%E6%A6%82%E8%BF%B0/</url>
|
|
<content><![CDATA[<p>G1: The Garbage-First Collector, 垃圾回收优先的垃圾回收器，目标是用户多核 cpu 和大内存的机器，最大的特点就是可预测的停顿时间，官方给出的介绍是提供一个用户在大的堆内存情况下一个低延迟表现的解决方案，通常是 6GB 及以上的堆大小，有低于 0.5 秒稳定的可预测的停顿时间。</p>
|
|
<p>这里主要介绍这个比较新的垃圾回收器，在 G1 之前的垃圾回收器都是基于如下图的内存结构分布，有新生代，老年代和永久代（jdk8 之前），然后G1 往前的那些垃圾回收器都有个分代，比如 serial，parallel 等，一般有个应用的组合，最初的 serial 和 serial old，因为新生代和老年代的收集方式不太一样，新生代主要是标记复制，所以有 eden 跟两个 survival区，老年代一般用标记整理方式，而 G1 对这个不太一样。<br><img data-src="https://i.loli.net/2020/02/09/jOVs2AlphzwyF5c.jpg"><br>看一下 G1 的内存分布<br><img data-src="https://i.loli.net/2020/02/09/Yr1tGiWp4mAZSzB.jpg"><br>可以看到这有很大的不同，G1 通过将内存分成大小相等的 region，每个region是存在于一个连续的虚拟内存范围，对于某个 region 来说其角色是类似于原来的收集器的Eden、Survivor、Old Generation，这个具体在代码层面</p>
|
|
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="comment">// We encode the value of the heap region type so the generation can be</span></span><br><span class="line"> <span class="comment">// determined quickly. The tag is split into two parts:</span></span><br><span class="line"> <span class="comment">//</span></span><br><span class="line"> <span class="comment">// major type (young, old, humongous, archive) : top N-1 bits</span></span><br><span class="line"> <span class="comment">// minor type (eden / survivor, starts / cont hum, etc.) : bottom 1 bit</span></span><br><span class="line"> <span class="comment">//</span></span><br><span class="line"> <span class="comment">// If there's need to increase the number of minor types in the</span></span><br><span class="line"> <span class="comment">// future, we'll have to increase the size of the latter and hence</span></span><br><span class="line"> <span class="comment">// decrease the size of the former.</span></span><br><span class="line"> <span class="comment">//</span></span><br><span class="line"> <span class="comment">// 00000 0 [ 0] Free</span></span><br><span class="line"> <span class="comment">//</span></span><br><span class="line"> <span class="comment">// 00001 0 [ 2] Young Mask</span></span><br><span class="line"> <span class="comment">// 00001 0 [ 2] Eden</span></span><br><span class="line"> <span class="comment">// 00001 1 [ 3] Survivor</span></span><br><span class="line"> <span class="comment">//</span></span><br><span class="line"> <span class="comment">// 00010 0 [ 4] Humongous Mask</span></span><br><span class="line"> <span class="comment">// 00100 0 [ 8] Pinned Mask</span></span><br><span class="line"> <span class="comment">// 00110 0 [12] Starts Humongous</span></span><br><span class="line"> <span class="comment">// 00110 1 [13] Continues Humongous</span></span><br><span class="line"> <span class="comment">//</span></span><br><span class="line"> <span class="comment">// 01000 0 [16] Old Mask</span></span><br><span class="line"> <span class="comment">//</span></span><br><span class="line"> <span class="comment">// 10000 0 [32] Archive Mask</span></span><br><span class="line"> <span class="comment">// 11100 0 [56] Open Archive</span></span><br><span class="line"> <span class="comment">// 11100 1 [57] Closed Archive</span></span><br><span class="line"> <span class="comment">//</span></span><br><span class="line"> <span class="keyword">typedef</span> <span class="keyword">enum</span> {</span><br><span class="line"> FreeTag = <span class="number">0</span>,</span><br><span class="line"></span><br><span class="line"> YoungMask = <span class="number">2</span>,</span><br><span class="line"> EdenTag = YoungMask,</span><br><span class="line"> SurvTag = YoungMask + <span class="number">1</span>,</span><br><span class="line"></span><br><span class="line"> HumongousMask = <span class="number">4</span>,</span><br><span class="line"> PinnedMask = <span class="number">8</span>,</span><br><span class="line"> StartsHumongousTag = HumongousMask | PinnedMask,</span><br><span class="line"> ContinuesHumongousTag = HumongousMask | PinnedMask + <span class="number">1</span>,</span><br><span class="line"></span><br><span class="line"> OldMask = <span class="number">16</span>,</span><br><span class="line"> OldTag = OldMask,</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Archive regions are regions with immutable content (i.e. not reclaimed, and</span></span><br><span class="line"> <span class="comment">// not allocated into during regular operation). They differ in the kind of references</span></span><br><span class="line"> <span class="comment">// allowed for the contained objects:</span></span><br><span class="line"> <span class="comment">// - Closed archive regions form a separate self-contained (closed) object graph</span></span><br><span class="line"> <span class="comment">// within the set of all of these regions. No references outside of closed</span></span><br><span class="line"> <span class="comment">// archive regions are allowed.</span></span><br><span class="line"> <span class="comment">// - Open archive regions have no restrictions on the references of their objects.</span></span><br><span class="line"> <span class="comment">// Objects within these regions are allowed to have references to objects</span></span><br><span class="line"> <span class="comment">// contained in any other kind of regions.</span></span><br><span class="line"> ArchiveMask = <span class="number">32</span>,</span><br><span class="line"> OpenArchiveTag = ArchiveMask | PinnedMask | OldMask,</span><br><span class="line"> ClosedArchiveTag = ArchiveMask | PinnedMask | OldMask + <span class="number">1</span></span><br><span class="line"> } Tag;</span><br></pre></td></tr></table></figure>
|
|
|
|
<p><code>hotspot/share/gc/g1/heapRegionType.hpp</code></p>
|
|
<p>当执行垃圾收集时，G1以类似于CMS收集器的方式运行。 G1执行并发全局标记阶段，以确定整个堆中对象的存活性。标记阶段完成后，G1知道哪些region是基本空的。它首先收集这些region，通常会产生大量的可用空间。这就是为什么这种垃圾收集方法称为“垃圾优先”的原因。顾名思义，G1将其收集和压缩活动集中在可能充满可回收对象（即垃圾）的堆区域。 G1使用暂停预测模型来满足用户定义的暂停时间目标，并根据指定的暂停时间目标选择要收集的区域数。</p>
|
|
<p>由G1标识为可回收的区域是使用撤离的方式(Evacuation)。 G1将对象从堆的一个或多个区域复制到堆上的单个区域，并在此过程中压缩并释放内存。撤离是在多处理器上并行执行的，以减少暂停时间并增加吞吐量。因此，对于每次垃圾收集，G1都在用户定义的暂停时间内连续工作以减少碎片。这是优于前面两种方法的。 CMS（并发标记扫描）垃圾收集器不进行压缩。 ParallelOld垃圾回收仅执行整个堆压缩，这导致相当长的暂停时间。</p>
|
|
<p>需要重点注意的是，G1不是实时收集器。它很有可能达到设定的暂停时间目标，但并非绝对确定。 G1根据先前收集的数据，估算在用户指定的目标时间内可以收集多少个区域。因此，收集器具有收集区域成本的合理准确的模型，并且收集器使用此模型来确定要收集哪些和多少个区域，同时保持在暂停时间目标之内。</p>
|
|
<p>注意：G1同时具有并发（与应用程序线程一起运行，例如优化，标记，清理）和并行（多线程，例如stw）阶段。Full GC仍然是单线程的，但是如果正确调优，您的应用程序应该可以避免Full GC。</p>
|
|
<p>在前面那篇中在代码层面简单的了解了这个可预测时间的过程，这也是 G1 的一大特点。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>JVM</category>
|
|
<category>C++</category>
|
|
<category>GC</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>C++</tag>
|
|
<tag>JVM</tag>
|
|
<tag>G1</tag>
|
|
<tag>GC</tag>
|
|
<tag>Garbage-First Collector</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Headscale初体验以及踩坑记</title>
|
|
<url>/2023/01/22/Headscale%E5%88%9D%E4%BD%93%E9%AA%8C%E4%BB%A5%E5%8F%8A%E8%B8%A9%E5%9D%91%E8%AE%B0/</url>
|
|
<content><![CDATA[<p>最近或者说很久以前就想着能够把几个散装服务器以及家里的网络连起来，譬如一些remote desktop的访问，之前搞了下frp，因为家里电脑没怎么注意安全性就被搞了一下，所以还是想用相对更安全的方式，比如限定ip和端口进行访问，但是感觉ip也不固定就比较难搞，后来看到了 <code>Tailscale</code> 和 <code>Headscale</code> 的方式，就想着试试看，没想到一开始就踩了几个比较莫名其妙的坑。<br>可以按<a href="https://github.com/juanfont/headscale/blob/main/docs/running-headscale-linux.md">官方文档</a>去搭建，也可以在网上找一些其他人搭建的教程。我碰到的主要是关于配置文件的问题</p>
|
|
<h3 id="第一个问题"><a href="#第一个问题" class="headerlink" title="第一个问题"></a>第一个问题</h3><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">Error initializing error="failed to read or create private key: failed to save private key to disk: open /etc/headscale/private.key: read-only file system"</span><br></pre></td></tr></table></figure>
|
|
<p>其实一开始看到这个我都有点懵了，咋回事呢，<code>read-only file system</code>一般有可能是文件系统出问题了，不可写入，需要重启或者修改挂载方式，被这个错误的错误日志给误导了，后面才知道是配置文件，在另一个教程中也有个类似的回复，一开始没注意，其实就是同一个问题。<br>默认的配置文件是这样的</p>
|
|
<figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="meta">---</span></span><br><span class="line"><span class="comment"># headscale will look for a configuration file named `config.yaml` (or `config.json`) in the following order:</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># - `/etc/headscale`</span></span><br><span class="line"><span class="comment"># - `~/.headscale`</span></span><br><span class="line"><span class="comment"># - current working directory</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># The url clients will connect to.</span></span><br><span class="line"><span class="comment"># Typically this will be a domain like:</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># https://myheadscale.example.com:443</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="attr">server_url:</span> <span class="string">http://127.0.0.1:8080</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Address to listen to / bind to on the server</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># For production:</span></span><br><span class="line"><span class="comment"># listen_addr: 0.0.0.0:8080</span></span><br><span class="line"><span class="attr">listen_addr:</span> <span class="number">127.0</span><span class="number">.0</span><span class="number">.1</span><span class="string">:8080</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Address to listen to /metrics, you may want</span></span><br><span class="line"><span class="comment"># to keep this endpoint private to your internal</span></span><br><span class="line"><span class="comment"># network</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="attr">metrics_listen_addr:</span> <span class="number">127.0</span><span class="number">.0</span><span class="number">.1</span><span class="string">:9090</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Address to listen for gRPC.</span></span><br><span class="line"><span class="comment"># gRPC is used for controlling a headscale server</span></span><br><span class="line"><span class="comment"># remotely with the CLI</span></span><br><span class="line"><span class="comment"># <span class="doctag">Note:</span> Remote access _only_ works if you have</span></span><br><span class="line"><span class="comment"># valid certificates.</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># For production:</span></span><br><span class="line"><span class="comment"># grpc_listen_addr: 0.0.0.0:50443</span></span><br><span class="line"><span class="attr">grpc_listen_addr:</span> <span class="number">127.0</span><span class="number">.0</span><span class="number">.1</span><span class="string">:50443</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Allow the gRPC admin interface to run in INSECURE</span></span><br><span class="line"><span class="comment"># mode. This is not recommended as the traffic will</span></span><br><span class="line"><span class="comment"># be unencrypted. Only enable if you know what you</span></span><br><span class="line"><span class="comment"># are doing.</span></span><br><span class="line"><span class="attr">grpc_allow_insecure:</span> <span class="literal">false</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Private key used to encrypt the traffic between headscale</span></span><br><span class="line"><span class="comment"># and Tailscale clients.</span></span><br><span class="line"><span class="comment"># The private key file will be autogenerated if it's missing.</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># For production:</span></span><br><span class="line"><span class="comment"># /var/lib/headscale/private.key</span></span><br><span class="line"><span class="attr">private_key_path:</span> <span class="string">./private.key</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># The Noise section includes specific configuration for the</span></span><br><span class="line"><span class="comment"># TS2021 Noise protocol</span></span><br><span class="line"><span class="attr">noise:</span></span><br><span class="line"> <span class="comment"># The Noise private key is used to encrypt the</span></span><br><span class="line"> <span class="comment"># traffic between headscale and Tailscale clients when</span></span><br><span class="line"> <span class="comment"># using the new Noise-based protocol. It must be different</span></span><br><span class="line"> <span class="comment"># from the legacy private key.</span></span><br><span class="line"> <span class="comment">#</span></span><br><span class="line"> <span class="comment"># For production:</span></span><br><span class="line"> <span class="comment"># private_key_path: /var/lib/headscale/noise_private.key</span></span><br><span class="line"> <span class="attr">private_key_path:</span> <span class="string">./noise_private.key</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># List of IP prefixes to allocate tailaddresses from.</span></span><br><span class="line"><span class="comment"># Each prefix consists of either an IPv4 or IPv6 address,</span></span><br><span class="line"><span class="comment"># and the associated prefix length, delimited by a slash.</span></span><br><span class="line"><span class="comment"># While this looks like it can take arbitrary values, it</span></span><br><span class="line"><span class="comment"># needs to be within IP ranges supported by the Tailscale</span></span><br><span class="line"><span class="comment"># client.</span></span><br><span class="line"><span class="comment"># IPv6: https://github.com/tailscale/tailscale/blob/22ebb25e833264f58d7c3f534a8b166894a89536/net/tsaddr/tsaddr.go#LL81C52-L81C71</span></span><br><span class="line"><span class="comment"># IPv4: https://github.com/tailscale/tailscale/blob/22ebb25e833264f58d7c3f534a8b166894a89536/net/tsaddr/tsaddr.go#L33</span></span><br><span class="line"><span class="attr">ip_prefixes:</span></span><br><span class="line"> <span class="bullet">-</span> <span class="string">fd7a:115c:a1e0::/48</span></span><br><span class="line"> <span class="bullet">-</span> <span class="number">100.64</span><span class="number">.0</span><span class="number">.0</span><span class="string">/10</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># DERP is a relay system that Tailscale uses when a direct</span></span><br><span class="line"><span class="comment"># connection cannot be established.</span></span><br><span class="line"><span class="comment"># https://tailscale.com/blog/how-tailscale-works/#encrypted-tcp-relays-derp</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># headscale needs a list of DERP servers that can be presented</span></span><br><span class="line"><span class="comment"># to the clients.</span></span><br><span class="line"><span class="attr">derp:</span></span><br><span class="line"> <span class="attr">server:</span></span><br><span class="line"> <span class="comment"># If enabled, runs the embedded DERP server and merges it into the rest of the DERP config</span></span><br><span class="line"> <span class="comment"># The Headscale server_url defined above MUST be using https, DERP requires TLS to be in place</span></span><br><span class="line"> <span class="attr">enabled:</span> <span class="literal">false</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># Region ID to use for the embedded DERP server.</span></span><br><span class="line"> <span class="comment"># The local DERP prevails if the region ID collides with other region ID coming from</span></span><br><span class="line"> <span class="comment"># the regular DERP config.</span></span><br><span class="line"> <span class="attr">region_id:</span> <span class="number">999</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># Region code and name are displayed in the Tailscale UI to identify a DERP region</span></span><br><span class="line"> <span class="attr">region_code:</span> <span class="string">"headscale"</span></span><br><span class="line"> <span class="attr">region_name:</span> <span class="string">"Headscale Embedded DERP"</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># Listens over UDP at the configured address for STUN connections - to help with NAT traversal.</span></span><br><span class="line"> <span class="comment"># When the embedded DERP server is enabled stun_listen_addr MUST be defined.</span></span><br><span class="line"> <span class="comment">#</span></span><br><span class="line"> <span class="comment"># For more details on how this works, check this great article: https://tailscale.com/blog/how-tailscale-works/</span></span><br><span class="line"> <span class="attr">stun_listen_addr:</span> <span class="string">"0.0.0.0:3478"</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># List of externally available DERP maps encoded in JSON</span></span><br><span class="line"> <span class="attr">urls:</span></span><br><span class="line"> <span class="bullet">-</span> <span class="string">https://controlplane.tailscale.com/derpmap/default</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># Locally available DERP map files encoded in YAML</span></span><br><span class="line"> <span class="comment">#</span></span><br><span class="line"> <span class="comment"># This option is mostly interesting for people hosting</span></span><br><span class="line"> <span class="comment"># their own DERP servers:</span></span><br><span class="line"> <span class="comment"># https://tailscale.com/kb/1118/custom-derp-servers/</span></span><br><span class="line"> <span class="comment">#</span></span><br><span class="line"> <span class="comment"># paths:</span></span><br><span class="line"> <span class="comment"># - /etc/headscale/derp-example.yaml</span></span><br><span class="line"> <span class="attr">paths:</span> []</span><br><span class="line"></span><br><span class="line"> <span class="comment"># If enabled, a worker will be set up to periodically</span></span><br><span class="line"> <span class="comment"># refresh the given sources and update the derpmap</span></span><br><span class="line"> <span class="comment"># will be set up.</span></span><br><span class="line"> <span class="attr">auto_update_enabled:</span> <span class="literal">true</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># How often should we check for DERP updates?</span></span><br><span class="line"> <span class="attr">update_frequency:</span> <span class="string">24h</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Disables the automatic check for headscale updates on startup</span></span><br><span class="line"><span class="attr">disable_check_updates:</span> <span class="literal">false</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Time before an inactive ephemeral node is deleted?</span></span><br><span class="line"><span class="attr">ephemeral_node_inactivity_timeout:</span> <span class="string">30m</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Period to check for node updates within the tailnet. A value too low will severely affect</span></span><br><span class="line"><span class="comment"># CPU consumption of Headscale. A value too high (over 60s) will cause problems</span></span><br><span class="line"><span class="comment"># for the nodes, as they won't get updates or keep alive messages frequently enough.</span></span><br><span class="line"><span class="comment"># In case of doubts, do not touch the default 10s.</span></span><br><span class="line"><span class="attr">node_update_check_interval:</span> <span class="string">10s</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># SQLite config</span></span><br><span class="line"><span class="attr">db_type:</span> <span class="string">sqlite3</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># For production:</span></span><br><span class="line"><span class="comment"># db_path: /var/lib/headscale/db.sqlite</span></span><br><span class="line"><span class="attr">db_path:</span> <span class="string">./db.sqlite</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># # Postgres config</span></span><br><span class="line"><span class="comment"># If using a Unix socket to connect to Postgres, set the socket path in the 'host' field and leave 'port' blank.</span></span><br><span class="line"><span class="comment"># db_type: postgres</span></span><br><span class="line"><span class="comment"># db_host: localhost</span></span><br><span class="line"><span class="comment"># db_port: 5432</span></span><br><span class="line"><span class="comment"># db_name: headscale</span></span><br><span class="line"><span class="comment"># db_user: foo</span></span><br><span class="line"><span class="comment"># db_pass: bar</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># If other 'sslmode' is required instead of 'require(true)' and 'disabled(false)', set the 'sslmode' you need</span></span><br><span class="line"><span class="comment"># in the 'db_ssl' field. Refers to https://www.postgresql.org/docs/current/libpq-ssl.html Table 34.1.</span></span><br><span class="line"><span class="comment"># db_ssl: false</span></span><br><span class="line"></span><br><span class="line"><span class="comment">### TLS configuration</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment">## Let's encrypt / ACME</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># headscale supports automatically requesting and setting up</span></span><br><span class="line"><span class="comment"># TLS for a domain with Let's Encrypt.</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># URL to ACME directory</span></span><br><span class="line"><span class="attr">acme_url:</span> <span class="string">https://acme-v02.api.letsencrypt.org/directory</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Email to register with ACME provider</span></span><br><span class="line"><span class="attr">acme_email:</span> <span class="string">""</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Domain name to request a TLS certificate for:</span></span><br><span class="line"><span class="attr">tls_letsencrypt_hostname:</span> <span class="string">""</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Path to store certificates and metadata needed by</span></span><br><span class="line"><span class="comment"># letsencrypt</span></span><br><span class="line"><span class="comment"># For production:</span></span><br><span class="line"><span class="comment"># tls_letsencrypt_cache_dir: /var/lib/headscale/cache</span></span><br><span class="line"><span class="attr">tls_letsencrypt_cache_dir:</span> <span class="string">./cache</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Type of ACME challenge to use, currently supported types:</span></span><br><span class="line"><span class="comment"># HTTP-01 or TLS-ALPN-01</span></span><br><span class="line"><span class="comment"># See [docs/tls.md](docs/tls.md) for more information</span></span><br><span class="line"><span class="attr">tls_letsencrypt_challenge_type:</span> <span class="string">HTTP-01</span></span><br><span class="line"><span class="comment"># When HTTP-01 challenge is chosen, letsencrypt must set up a</span></span><br><span class="line"><span class="comment"># verification endpoint, and it will be listening on:</span></span><br><span class="line"><span class="comment"># :http = port 80</span></span><br><span class="line"><span class="attr">tls_letsencrypt_listen:</span> <span class="string">":http"</span></span><br><span class="line"></span><br><span class="line"><span class="comment">## Use already defined certificates:</span></span><br><span class="line"><span class="attr">tls_cert_path:</span> <span class="string">""</span></span><br><span class="line"><span class="attr">tls_key_path:</span> <span class="string">""</span></span><br><span class="line"></span><br><span class="line"><span class="attr">log:</span></span><br><span class="line"> <span class="comment"># Output formatting for logs: text or json</span></span><br><span class="line"> <span class="attr">format:</span> <span class="string">text</span></span><br><span class="line"> <span class="attr">level:</span> <span class="string">info</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Path to a file containg ACL policies.</span></span><br><span class="line"><span class="comment"># ACLs can be defined as YAML or HUJSON.</span></span><br><span class="line"><span class="comment"># https://tailscale.com/kb/1018/acls/</span></span><br><span class="line"><span class="attr">acl_policy_path:</span> <span class="string">""</span></span><br><span class="line"></span><br><span class="line"><span class="comment">## DNS</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># headscale supports Tailscale's DNS configuration and MagicDNS.</span></span><br><span class="line"><span class="comment"># Please have a look to their KB to better understand the concepts:</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># - https://tailscale.com/kb/1054/dns/</span></span><br><span class="line"><span class="comment"># - https://tailscale.com/kb/1081/magicdns/</span></span><br><span class="line"><span class="comment"># - https://tailscale.com/blog/2021-09-private-dns-with-magicdns/</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="attr">dns_config:</span></span><br><span class="line"> <span class="comment"># Whether to prefer using Headscale provided DNS or use local.</span></span><br><span class="line"> <span class="attr">override_local_dns:</span> <span class="literal">true</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># List of DNS servers to expose to clients.</span></span><br><span class="line"> <span class="attr">nameservers:</span></span><br><span class="line"> <span class="bullet">-</span> <span class="number">1.1</span><span class="number">.1</span><span class="number">.1</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># NextDNS (see https://tailscale.com/kb/1218/nextdns/).</span></span><br><span class="line"> <span class="comment"># "abc123" is example NextDNS ID, replace with yours.</span></span><br><span class="line"> <span class="comment">#</span></span><br><span class="line"> <span class="comment"># With metadata sharing:</span></span><br><span class="line"> <span class="comment"># nameservers:</span></span><br><span class="line"> <span class="comment"># - https://dns.nextdns.io/abc123</span></span><br><span class="line"> <span class="comment">#</span></span><br><span class="line"> <span class="comment"># Without metadata sharing:</span></span><br><span class="line"> <span class="comment"># nameservers:</span></span><br><span class="line"> <span class="comment"># - 2a07:a8c0::ab:c123</span></span><br><span class="line"> <span class="comment"># - 2a07:a8c1::ab:c123</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># Split DNS (see https://tailscale.com/kb/1054/dns/),</span></span><br><span class="line"> <span class="comment"># list of search domains and the DNS to query for each one.</span></span><br><span class="line"> <span class="comment">#</span></span><br><span class="line"> <span class="comment"># restricted_nameservers:</span></span><br><span class="line"> <span class="comment"># foo.bar.com:</span></span><br><span class="line"> <span class="comment"># - 1.1.1.1</span></span><br><span class="line"> <span class="comment"># darp.headscale.net:</span></span><br><span class="line"> <span class="comment"># - 1.1.1.1</span></span><br><span class="line"> <span class="comment"># - 8.8.8.8</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># Search domains to inject.</span></span><br><span class="line"> <span class="attr">domains:</span> []</span><br><span class="line"></span><br><span class="line"> <span class="comment"># Extra DNS records</span></span><br><span class="line"> <span class="comment"># so far only A-records are supported (on the tailscale side)</span></span><br><span class="line"> <span class="comment"># See https://github.com/juanfont/headscale/blob/main/docs/dns-records.md#Limitations</span></span><br><span class="line"> <span class="comment"># extra_records:</span></span><br><span class="line"> <span class="comment"># - name: "grafana.myvpn.example.com"</span></span><br><span class="line"> <span class="comment"># type: "A"</span></span><br><span class="line"> <span class="comment"># value: "100.64.0.3"</span></span><br><span class="line"> <span class="comment">#</span></span><br><span class="line"> <span class="comment"># # you can also put it in one line</span></span><br><span class="line"> <span class="comment"># - { name: "prometheus.myvpn.example.com", type: "A", value: "100.64.0.3" }</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># Whether to use [MagicDNS](https://tailscale.com/kb/1081/magicdns/).</span></span><br><span class="line"> <span class="comment"># Only works if there is at least a nameserver defined.</span></span><br><span class="line"> <span class="attr">magic_dns:</span> <span class="literal">true</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># Defines the base domain to create the hostnames for MagicDNS.</span></span><br><span class="line"> <span class="comment"># `base_domain` must be a FQDNs, without the trailing dot.</span></span><br><span class="line"> <span class="comment"># The FQDN of the hosts will be</span></span><br><span class="line"> <span class="comment"># `hostname.user.base_domain` (e.g., _myhost.myuser.example.com_).</span></span><br><span class="line"> <span class="attr">base_domain:</span> <span class="string">example.com</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Unix socket used for the CLI to connect without authentication</span></span><br><span class="line"><span class="comment"># <span class="doctag">Note:</span> for production you will want to set this to something like:</span></span><br><span class="line"><span class="comment"># unix_socket: /var/run/headscale.sock</span></span><br><span class="line"><span class="attr">unix_socket:</span> <span class="string">./headscale.sock</span></span><br><span class="line"><span class="attr">unix_socket_permission:</span> <span class="string">"0770"</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># headscale supports experimental OpenID connect support,</span></span><br><span class="line"><span class="comment"># it is still being tested and might have some bugs, please</span></span><br><span class="line"><span class="comment"># help us test it.</span></span><br><span class="line"><span class="comment"># OpenID Connect</span></span><br><span class="line"><span class="comment"># oidc:</span></span><br><span class="line"><span class="comment"># only_start_if_oidc_is_available: true</span></span><br><span class="line"><span class="comment"># issuer: "https://your-oidc.issuer.com/path"</span></span><br><span class="line"><span class="comment"># client_id: "your-oidc-client-id"</span></span><br><span class="line"><span class="comment"># client_secret: "your-oidc-client-secret"</span></span><br><span class="line"><span class="comment"># # Alternatively, set `client_secret_path` to read the secret from the file.</span></span><br><span class="line"><span class="comment"># # It resolves environment variables, making integration to systemd's</span></span><br><span class="line"><span class="comment"># # `LoadCredential` straightforward:</span></span><br><span class="line"><span class="comment"># client_secret_path: "${CREDENTIALS_DIRECTORY}/oidc_client_secret"</span></span><br><span class="line"><span class="comment"># # client_secret and client_secret_path are mutually exclusive.</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># Customize the scopes used in the OIDC flow, defaults to "openid", "profile" and "email" and add custom query</span></span><br><span class="line"><span class="comment"># parameters to the Authorize Endpoint request. Scopes default to "openid", "profile" and "email".</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># scope: ["openid", "profile", "email", "custom"]</span></span><br><span class="line"><span class="comment"># extra_params:</span></span><br><span class="line"><span class="comment"># domain_hint: example.com</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># List allowed principal domains and/or users. If an authenticated user's domain is not in this list, the</span></span><br><span class="line"><span class="comment"># authentication request will be rejected.</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># allowed_domains:</span></span><br><span class="line"><span class="comment"># - example.com</span></span><br><span class="line"><span class="comment"># Groups from keycloak have a leading '/'</span></span><br><span class="line"><span class="comment"># allowed_groups:</span></span><br><span class="line"><span class="comment"># - /headscale</span></span><br><span class="line"><span class="comment"># allowed_users:</span></span><br><span class="line"><span class="comment"># - alice@example.com</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># If `strip_email_domain` is set to `true`, the domain part of the username email address will be removed.</span></span><br><span class="line"><span class="comment"># This will transform `first-name.last-name@example.com` to the user `first-name.last-name`</span></span><br><span class="line"><span class="comment"># If `strip_email_domain` is set to `false` the domain part will NOT be removed resulting to the following</span></span><br><span class="line"><span class="comment"># user: `first-name.last-name.example.com`</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># strip_email_domain: true</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Logtail configuration</span></span><br><span class="line"><span class="comment"># Logtail is Tailscales logging and auditing infrastructure, it allows the control panel</span></span><br><span class="line"><span class="comment"># to instruct tailscale nodes to log their activity to a remote server.</span></span><br><span class="line"><span class="attr">logtail:</span></span><br><span class="line"> <span class="comment"># Enable logtail for this headscales clients.</span></span><br><span class="line"> <span class="comment"># As there is currently no support for overriding the log server in headscale, this is</span></span><br><span class="line"> <span class="comment"># disabled by default. Enabling this will make your clients send logs to Tailscale Inc.</span></span><br><span class="line"> <span class="attr">enabled:</span> <span class="literal">false</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Enabling this option makes devices prefer a random port for WireGuard traffic over the</span></span><br><span class="line"><span class="comment"># default static port 41641. This option is intended as a workaround for some buggy</span></span><br><span class="line"><span class="comment"># firewall devices. See https://tailscale.com/kb/1181/firewalls/ for more information.</span></span><br><span class="line"><span class="attr">randomize_client_port:</span> <span class="literal">false</span></span><br></pre></td></tr></table></figure>
|
|
|
|
<p>问题就是出在几个文件路径的配置，默认都是当前目录，也就是headscale的可执行文件所在目录，需要按它配置说明中的生产配置进行修改</p>
|
|
<figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="comment"># For production:</span></span><br><span class="line"><span class="comment"># /var/lib/headscale/private.key</span></span><br><span class="line"><span class="attr">private_key_path:</span> <span class="string">/var/lib/headscale/private.key</span></span><br></pre></td></tr></table></figure>
|
|
<p>直接改成绝对路径就好了，还有两个文件路径<br>另一个也是个秘钥的路径问题</p>
|
|
<figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="attr">noise:</span></span><br><span class="line"> <span class="comment"># The Noise private key is used to encrypt the</span></span><br><span class="line"> <span class="comment"># traffic between headscale and Tailscale clients when</span></span><br><span class="line"> <span class="comment"># using the new Noise-based protocol. It must be different</span></span><br><span class="line"> <span class="comment"># from the legacy private key.</span></span><br><span class="line"> <span class="comment">#</span></span><br><span class="line"> <span class="comment"># For production:</span></span><br><span class="line"> <span class="comment"># private_key_path: /var/lib/headscale/noise_private.key</span></span><br><span class="line"> <span class="attr">private_key_path:</span> <span class="string">/var/lib/headscale/noise_private.key</span></span><br></pre></td></tr></table></figure>
|
|
<h3 id="第二个问题"><a href="#第二个问题" class="headerlink" title="第二个问题"></a>第二个问题</h3><p>这个问题也是一种误导，<br>错误信息是</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">Error initializing error="unable to open database file: out of memory (14)"</span><br></pre></td></tr></table></figure>
|
|
<p>这就是个文件，内存也完全没有被占满的迹象，原来也是文件路径的问题</p>
|
|
<figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="comment"># For production:</span></span><br><span class="line"><span class="comment"># db_path: /var/lib/headscale/db.sqlite</span></span><br><span class="line"><span class="attr">db_path:</span> <span class="string">/var/lib/headscale/db.sqlite</span></span><br></pre></td></tr></table></figure>
|
|
<p>都改成绝对路径就可以了，然后这里还有个就是要对<code>/var/lib/headscale/</code>和<code>/etc/headscale/</code>等路径赋予headscale用户权限，有时候对这类问题的排查真的蛮头疼，日志报错都不是真实的错误信息，开源项目对这些错误的提示真的也需要优化，后续的譬如mac也加入节点等后面再开篇讲</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>headscale</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>headscale</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Headscale下篇-自定义中转derper</title>
|
|
<url>/2024/04/14/Headscale%E4%B8%8B%E7%AF%87-%E8%87%AA%E5%AE%9A%E4%B9%89%E4%B8%AD%E8%BD%ACderper/</url>
|
|
<content><![CDATA[<p>derper是 tailscale 已经开源的 tailscale 中转服务，可以进行自己搭建，不然就是使用 tailscale 提供的 derper 节点，我的体验是在晚上高峰期会比较卡，所以有能力可以自建的话也是比较推荐的，不过相对比前面的设置会麻烦点</p>
|
|
<p>第一个就是默认这个是需要 https 的，所以需要给 derper 服务申请个域名，或者有了域名就加个解析，比如 dp.hello.com，申请完域名就需要申请证书了，这个刚好就学习下使用 acme.sh这个工具，还是非常不错的，我用的是腾讯云的域名解析服务，其实是收购的 dnspod 的服务，acme.sh 安装就很简单，一条命令</p>
|
|
<p>`</p>
|
|
<figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">curl https://get.acme.sh | sh</span><br></pre></td></tr></table></figure>
|
|
<p>然后成功了之后就要把<code>.zshrc</code> 生效下，默认会把脚本路径加入到当前的 shell 的 rc 文件里，然后就是申请证书，但是有两种方式，一种就是在域名指定的 ip 机器对应的访问路径下放置验证的文件</p>
|
|
<figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">acme.sh --issue -d domain.tld -d www.domain.tld --webroot /home/wwwroot/domain.tld/</span><br></pre></td></tr></table></figure>
|
|
<p>比如这样指定 webroot，另一个中更方便的方式就是通过 dns 认证的方式，我这边以 dnspod 举例，这边要特别注意的是 dnspod 虽然被腾讯云收购，但是这个方式还是需要用 dnspod 原来自己的 api token 验证的</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/SbFDCl.png"></p>
|
|
<p>然后拿到生成的 id 跟 token 设置到环境变量里，需要保存好这个 token，后面是看不到了的</p>
|
|
<figure class="highlight bash"><table><tr><td class="code"><pre><span class="line"><span class="built_in">export</span> DP_Id=123456</span><br><span class="line"><span class="built_in">export</span> DP_Key=xxxxxx</span><br></pre></td></tr></table></figure>
|
|
<p>然后就可以用命令进行验证了</p>
|
|
<figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">acme.sh --issue --dns dns_dp -d xxx.hello.com</span><br></pre></td></tr></table></figure>
|
|
<p>这个是 dnspod 家的，其他的也可以在<a href="https://github.com/acmesh-official/acme.sh/wiki/dnsapi#dns_dp">这里</a>找到验证方式</p>
|
|
<p>申请完成后会把证书放在用户目录下的<code>.acme.sh</code> 目录中，记得把cer文件改成 crt，并且文件名只能是域名 xxx.hello.com.crt，我没试过其他的行不行，从别的文章里看到需要一致，然后就是启动 derper 了</p>
|
|
<p>我们通过 docker 的方式，</p>
|
|
<figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">sudo docker run --restart always \</span><br><span class="line"> --name derper -p 12345:12345 -p 3478:3478/udp \</span><br><span class="line"> -v /<span class="variable">$dir</span>/.acme.sh/xxxxx.hello.com_ecc/:/app/certs \</span><br><span class="line"> -v /run/tailscale/tailscaled.sock:/var/run/tailscale/tailscaled.sock \</span><br><span class="line"> -e DERP_CERT_MODE=manual \</span><br><span class="line"> -e DERP_ADDR=:12345 \</span><br><span class="line"> -e DERP_DOMAIN=xxxxx.hello.com \</span><br><span class="line"> -e DERP_VERIFY_CLIENTS=<span class="literal">true</span> \</span><br><span class="line"> -d ghcr.io/yangchuansheng/derper:latest</span><br></pre></td></tr></table></figure>
|
|
<p> 前一个端口 12345可以自己随意定，后面$dir改成自己的.acme.sh目录所在的路径，然后</p>
|
|
<p><code> -v /run/tailscale/tailscaled.sock:/var/run/tailscale/tailscaled.sock \</code> 这一行 和<code>-e DERP_VERIFY_CLIENTS=true \</code> 是为了能够让 derper 能够安全一些，就是在本机开一个 tailscale 客户端，连接接到 headscale 服务，可以参考前面的<a href="https://nicksxs.me/2023/07/09/headscale-%E6%B7%BB%E5%8A%A0%E8%8A%82%E7%82%B9/">添加节点</a>，然后把这个启动客户端的进程映射进docker里面，因为本身 derper 就没有特别的认证逻辑，所以就让同一个headscale服务的客户端进程来验证是属于同一个headscale服务的节点才允许加入中转，实际这样就跑起来了一个 derper 服务这样可以用 docker 命令查看日志</p>
|
|
<p><code>sudo docker logs -f derper</code> 查看是不是有问题，然后如果没有问题的话还不一定代表 derper 起来了，比较直接的方法就是直接打开域名加上端口的地址，比如 xxxx.hello.com:12345 这个,</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/J3SWNz.png" alt="image"></p>
|
|
<p>显示这个就代表服务已经正常启动运行了，然后就是需要修改 headscale 的配置文件了，首先要设置 derper 的配置文件，类似这样</p>
|
|
<figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">regions:</span><br><span class="line"> 100:</span><br><span class="line"> regionid: 100</span><br><span class="line"> regioncode: ahk </span><br><span class="line"> regionname: Aliyun Hongkong </span><br><span class="line"> nodes:</span><br><span class="line"> - name: 100a</span><br><span class="line"> regionid: 100</span><br><span class="line"> hostname: xxxxx.hello.com</span><br><span class="line"> ipv4:</span><br><span class="line"> stunport: 3478</span><br><span class="line"> stunonly: <span class="literal">false</span></span><br><span class="line"> derpport: 12345</span><br></pre></td></tr></table></figure>
|
|
<p>regionid表示区域 id，一个区域下可以后多个 node，node的 name 是节点的唯一识别码，ipv4 可以不填，有 hostname 就可以，然后stunonly 是 false 表示不只使用 stun还可以使用 derp</p>
|
|
<p>然后就是修改 headscale 自身的配置文件，把共用的 derper 配置改成</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/nlk7m9.png" alt="image"></p>
|
|
<p>上面的 urls 注释掉，后面的 paths 添加刚才的配置文件路径，这样就可以了，至于连接是不是走的这个</p>
|
|
<p>可以用 <code>tailscale netcheck</code> 查看最近的 derper 节点已经延迟，看到使用了自己的 derper 就对了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>headscale</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>headscale</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Headscale渐入佳境-路由设置</title>
|
|
<url>/2024/04/07/Headscale%E6%B8%90%E5%85%A5%E4%BD%B3%E5%A2%83-%E8%B7%AF%E7%94%B1%E8%AE%BE%E7%BD%AE/</url>
|
|
<content><![CDATA[<p><code>headscale</code> 在前面两篇<a href="https://nicksxs.me/2023/01/22/Headscale%E5%88%9D%E4%BD%93%E9%AA%8C%E4%BB%A5%E5%8F%8A%E8%B8%A9%E5%9D%91%E8%AE%B0/">初体验</a>和<a href="https://nicksxs.me/2023/07/09/headscale-%E6%B7%BB%E5%8A%A0%E8%8A%82%E7%82%B9/">添加节点</a> 主要是介绍了如何搭建和使用，但是这里有个很大的缺点，比如我在想要两地的网络能够连通，但是两地的网络内都不止一个机器，那想让他们连通就需要把它们都加入这个 <code>headscale</code> 所组成的虚拟局域网，并且每个都是这个虚拟局域网的ip，跟原来的 ip 地址不一样会比较难记忆，所以有没有办法不用这么麻烦呢，当然是有的，那就是 <code>headscale</code> 的路由功能。<br>原先比如我们有两个设备，在两地，通过headscale我把这两个异地节点都添加进去了，节点 A 的 ip比如是 <code>10.0.0.1</code>，另一个节点 B 是 <code>10.0.0.2</code>，在之前的状态下，我的节点 A 只能通过 <code>10.0.0.2</code> 来连接节点 B，如果节点 A 想要连接节点 B 所在地的局域网内部其他节点，我们就可以通过节点 B 的客户端登录的时候开启路由转发，这样节点 B 就会充当一个路由转发的功能，并且不需要通过10.0.0这个网段来访问，直接可以通过节点 B所在的局域网地址来访问，比如192.168.2.10，具体操作就是把客户端启动方式改成命令行的，主要就是 <code> --advertise-routes</code>, 指明需要转发的网段</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">tailscale up --advertise-routes=192.168.xx.0/24 --accept-dns=false --login-server=http://serverIp:serverPort --unattended </span><br></pre></td></tr></table></figure>
|
|
<p>前面的 <code>192.168.xx.0</code> 就是节点 B 所在局域网的网段，然后 <code>login-server</code> 是 <code>headscale</code> 的服务 ip 跟端口，这个本身客户端就会使用，另外这个还有一个前提的，需要开启服务器的端口转发，对于 Windows 其实默认就可以，Linux 的话就需要设置下</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_">$ </span><span class="language-bash"><span class="built_in">echo</span> <span class="string">'net.ipv4.ip_forward = 1'</span> | <span class="built_in">tee</span> /etc/sysctl.d/ipforwarding.conf</span></span><br><span class="line"><span class="meta prompt_">$ </span><span class="language-bash"><span class="built_in">echo</span> <span class="string">'net.ipv6.conf.all.forwarding = 1'</span> | <span class="built_in">tee</span> -a /etc/sysctl.d/ipforwarding.conf</span></span><br><span class="line"><span class="meta prompt_">$ </span><span class="language-bash">sysctl -p /etc/sysctl.d/ipforwarding.conf</span></span><br></pre></td></tr></table></figure>
|
|
|
|
<p>到这里我们只完成了第一步，第二步需要在 <code>headscale</code> 服务端启用这条路由配置<br>首先可以用 <code>sudo headscale routes</code> 查看当前有的路由<br><code>sudo headscale routes enable -h</code> 这里看下怎么启用，因为网上有些是用-i 指定哪一条路由的，但是我使用的这个版本是用-r 的<br><code>sudo headscale routes enable -r 1</code> 然后用这个命令启用就行了<br><img data-src="https://img.nicksxs.me/blog/h5Ihsg.jpg"><br><img data-src="https://img.nicksxs.me/blog/7vXV6A.jpg"><br>这样我们就能够很方便地在单个节点互连之后访问到对方局域网内的其他节点</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>headscale</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>headscale</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Headscale渐入佳境补充篇-自定义中转derper的证书问题</title>
|
|
<url>/2024/05/26/Headscale%E6%B8%90%E5%85%A5%E4%BD%B3%E5%A2%83%E8%A1%A5%E5%85%85%E7%AF%87-%E8%87%AA%E5%AE%9A%E4%B9%89%E4%B8%AD%E8%BD%ACderper%E7%9A%84%E8%AF%81%E4%B9%A6%E9%97%AE%E9%A2%98/</url>
|
|
<content><![CDATA[<p>之前自定义部署的derper在Mac端和Windows使用时没啥问题，但是在我想作为网关的Armbian小机器上一直会报 “x509: certificate signed by unknown authority”<br>错误，原因在于之前在通过acme生成证书以后我是直接把cer证书重命名成crt就映射到derper的docker目录里，对于Mac和Windows客户端应该是有兼容逻辑，而实际使用需要完整的证书去识别认证机构，否则就会被认为是自定义证书，在一些常规的证书认证逻辑里就会报上面的问题<br>这边需要先把生成的 <code>fullchain.cer</code> 完整证书转成crt格式的</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">openssl x509 -inform PEM -in fullchain.cer -out derper.demo.com.crt</span><br></pre></td></tr></table></figure>
|
|
<p>这边是PEM格式的证书，如果是DER的话就改一下</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">openssl x509 -inform DER -in certificate.cer -out certificate.crt</span><br></pre></td></tr></table></figure>
|
|
<p>转换完成后就重新启动下docker<br>这里的原因主要是fullchain.cer是包含了完整的证书链，而ca.cer只是包含用户证书，所以需要替换一下，这算是个小问题</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>headscale</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>headscale</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>JVM源码分析之G1垃圾收集器分析一</title>
|
|
<url>/2019/12/07/JVM-G1-Part-1/</url>
|
|
<content><![CDATA[<p>对 Java 的 gc 实现比较感兴趣，原先一般都是看周志明的书，但其实并没有讲具体的 gc 源码，而是把整个思路和流程讲解了一下<br>特别是 G1 的具体实现<br>一般对 G1 的理解其实就是把原先整块的新生代老年代分成了以 region 为单位的小块内存，简而言之，就是原先对新生代老年代的收集会涉及到整个代的堆内存空间，而G1 把它变成了更细致的小块内存<br>这带来了一个很明显的好处和一个很明显的坏处，好处是内存收集可以更灵活，耗时会变短，但整个收集的处理复杂度就变高了<br>目前看了一点点关于 G1 收集的预期时间相关的代码</p>
|
|
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="function">HeapWord* <span class="title">G1CollectedHeap::do_collection_pause</span><span class="params">(<span class="type">size_t</span> word_size,</span></span></span><br><span class="line"><span class="params"><span class="function"> uint gc_count_before,</span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">bool</span>* succeeded,</span></span></span><br><span class="line"><span class="params"><span class="function"> GCCause::Cause gc_cause)</span> </span>{</span><br><span class="line"> <span class="built_in">assert_heap_not_locked_and_not_at_safepoint</span>();</span><br><span class="line"> <span class="function">VM_G1CollectForAllocation <span class="title">op</span><span class="params">(word_size,</span></span></span><br><span class="line"><span class="params"><span class="function"> gc_count_before,</span></span></span><br><span class="line"><span class="params"><span class="function"> gc_cause,</span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="literal">false</span>, <span class="comment">/* should_initiate_conc_mark */</span></span></span></span><br><span class="line"><span class="params"><span class="function"> g1_policy()->max_pause_time_ms())</span></span>;</span><br><span class="line"> VMThread::<span class="built_in">execute</span>(&op);</span><br><span class="line"></span><br><span class="line"> HeapWord* result = op.<span class="built_in">result</span>();</span><br><span class="line"> <span class="type">bool</span> ret_succeeded = op.<span class="built_in">prologue_succeeded</span>() && op.<span class="built_in">pause_succeeded</span>();</span><br><span class="line"> <span class="built_in">assert</span>(result == <span class="literal">NULL</span> || ret_succeeded,</span><br><span class="line"> <span class="string">"the result should be NULL if the VM did not succeed"</span>);</span><br><span class="line"> *succeeded = ret_succeeded;</span><br><span class="line"></span><br><span class="line"> <span class="built_in">assert_heap_not_locked</span>();</span><br><span class="line"> <span class="keyword">return</span> result;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里就是收集时需要停顿的，其中<code>VMThread::execute(&op);</code>是具体执行的，真正执行的是<code>VM_G1CollectForAllocation::doit</code>方法</p>
|
|
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="type">void</span> <span class="title">VM_G1CollectForAllocation::doit</span><span class="params">()</span> </span>{</span><br><span class="line"> G1CollectedHeap* g1h = G1CollectedHeap::<span class="built_in">heap</span>();</span><br><span class="line"> <span class="built_in">assert</span>(!_should_initiate_conc_mark || g1h-><span class="built_in">should_do_concurrent_full_gc</span>(_gc_cause),</span><br><span class="line"> <span class="string">"only a GC locker, a System.gc(), stats update, whitebox, or a hum allocation induced GC should start a cycle"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (_word_size > <span class="number">0</span>) {</span><br><span class="line"> <span class="comment">// An allocation has been requested. So, try to do that first.</span></span><br><span class="line"> _result = g1h-><span class="built_in">attempt_allocation_at_safepoint</span>(_word_size,</span><br><span class="line"> <span class="literal">false</span> <span class="comment">/* expect_null_cur_alloc_region */</span>);</span><br><span class="line"> <span class="keyword">if</span> (_result != <span class="literal">NULL</span>) {</span><br><span class="line"> <span class="comment">// If we can successfully allocate before we actually do the</span></span><br><span class="line"> <span class="comment">// pause then we will consider this pause successful.</span></span><br><span class="line"> _pause_succeeded = <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="function">GCCauseSetter <span class="title">x</span><span class="params">(g1h, _gc_cause)</span></span>;</span><br><span class="line"> <span class="keyword">if</span> (_should_initiate_conc_mark) {</span><br><span class="line"> <span class="comment">// It's safer to read old_marking_cycles_completed() here, given</span></span><br><span class="line"> <span class="comment">// that noone else will be updating it concurrently. Since we'll</span></span><br><span class="line"> <span class="comment">// only need it if we're initiating a marking cycle, no point in</span></span><br><span class="line"> <span class="comment">// setting it earlier.</span></span><br><span class="line"> _old_marking_cycles_completed_before = g1h-><span class="built_in">old_marking_cycles_completed</span>();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// At this point we are supposed to start a concurrent cycle. We</span></span><br><span class="line"> <span class="comment">// will do so if one is not already in progress.</span></span><br><span class="line"> <span class="type">bool</span> res = g1h-><span class="built_in">g1_policy</span>()->force_initial_mark_if_outside_cycle(_gc_cause);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// The above routine returns true if we were able to force the</span></span><br><span class="line"> <span class="comment">// next GC pause to be an initial mark; it returns false if a</span></span><br><span class="line"> <span class="comment">// marking cycle is already in progress.</span></span><br><span class="line"> <span class="comment">//</span></span><br><span class="line"> <span class="comment">// If a marking cycle is already in progress just return and skip the</span></span><br><span class="line"> <span class="comment">// pause below - if the reason for requesting this initial mark pause</span></span><br><span class="line"> <span class="comment">// was due to a System.gc() then the requesting thread should block in</span></span><br><span class="line"> <span class="comment">// doit_epilogue() until the marking cycle is complete.</span></span><br><span class="line"> <span class="comment">//</span></span><br><span class="line"> <span class="comment">// If this initial mark pause was requested as part of a humongous</span></span><br><span class="line"> <span class="comment">// allocation then we know that the marking cycle must just have</span></span><br><span class="line"> <span class="comment">// been started by another thread (possibly also allocating a humongous</span></span><br><span class="line"> <span class="comment">// object) as there was no active marking cycle when the requesting</span></span><br><span class="line"> <span class="comment">// thread checked before calling collect() in</span></span><br><span class="line"> <span class="comment">// attempt_allocation_humongous(). Retrying the GC, in this case,</span></span><br><span class="line"> <span class="comment">// will cause the requesting thread to spin inside collect() until the</span></span><br><span class="line"> <span class="comment">// just started marking cycle is complete - which may be a while. So</span></span><br><span class="line"> <span class="comment">// we do NOT retry the GC.</span></span><br><span class="line"> <span class="keyword">if</span> (!res) {</span><br><span class="line"> <span class="built_in">assert</span>(_word_size == <span class="number">0</span>, <span class="string">"Concurrent Full GC/Humongous Object IM shouldn't be allocating"</span>);</span><br><span class="line"> <span class="keyword">if</span> (_gc_cause != GCCause::_g1_humongous_allocation) {</span><br><span class="line"> _should_retry_gc = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Try a partial collection of some kind.</span></span><br><span class="line"> _pause_succeeded = g1h-><span class="built_in">do_collection_pause_at_safepoint</span>(_target_pause_time_ms);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (_pause_succeeded) {</span><br><span class="line"> <span class="keyword">if</span> (_word_size > <span class="number">0</span>) {</span><br><span class="line"> <span class="comment">// An allocation had been requested. Do it, eventually trying a stronger</span></span><br><span class="line"> <span class="comment">// kind of GC.</span></span><br><span class="line"> _result = g1h-><span class="built_in">satisfy_failed_allocation</span>(_word_size, &_pause_succeeded);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="type">bool</span> should_upgrade_to_full = !g1h-><span class="built_in">should_do_concurrent_full_gc</span>(_gc_cause) &&</span><br><span class="line"> !g1h-><span class="built_in">has_regions_left_for_allocation</span>();</span><br><span class="line"> <span class="keyword">if</span> (should_upgrade_to_full) {</span><br><span class="line"> <span class="comment">// There has been a request to perform a GC to free some space. We have no</span></span><br><span class="line"> <span class="comment">// information on how much memory has been asked for. In case there are</span></span><br><span class="line"> <span class="comment">// absolutely no regions left to allocate into, do a maximally compacting full GC.</span></span><br><span class="line"> <span class="built_in">log_info</span>(gc, ergo)(<span class="string">"Attempting maximally compacting collection"</span>);</span><br><span class="line"> _pause_succeeded = g1h-><span class="built_in">do_full_collection</span>(<span class="literal">false</span>, <span class="comment">/* explicit gc */</span></span><br><span class="line"> <span class="literal">true</span> <span class="comment">/* clear_all_soft_refs */</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">guarantee</span>(_pause_succeeded, <span class="string">"Elevated collections during the safepoint must always succeed."</span>);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="built_in">assert</span>(_result == <span class="literal">NULL</span>, <span class="string">"invariant"</span>);</span><br><span class="line"> <span class="comment">// The only reason for the pause to not be successful is that, the GC locker is</span></span><br><span class="line"> <span class="comment">// active (or has become active since the prologue was executed). In this case</span></span><br><span class="line"> <span class="comment">// we should retry the pause after waiting for the GC locker to become inactive.</span></span><br><span class="line"> _should_retry_gc = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里可以看到核心的是<code>G1CollectedHeap::do_collection_pause_at_safepoint</code>这个方法，它带上了目标暂停时间的值</p>
|
|
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line">G1CollectedHeap::<span class="built_in">do_collection_pause_at_safepoint</span>(<span class="type">double</span> target_pause_time_ms) {</span><br><span class="line"> <span class="built_in">assert_at_safepoint_on_vm_thread</span>();</span><br><span class="line"> <span class="built_in">guarantee</span>(!<span class="built_in">is_gc_active</span>(), <span class="string">"collection is not reentrant"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (GCLocker::<span class="built_in">check_active_before_gc</span>()) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> _gc_timer_stw-><span class="built_in">register_gc_start</span>();</span><br><span class="line"></span><br><span class="line"> GCIdMark gc_id_mark;</span><br><span class="line"> _gc_tracer_stw-><span class="built_in">report_gc_start</span>(<span class="built_in">gc_cause</span>(), _gc_timer_stw-><span class="built_in">gc_start</span>());</span><br><span class="line"></span><br><span class="line"> <span class="function">SvcGCMarker <span class="title">sgcm</span><span class="params">(SvcGCMarker::MINOR)</span></span>;</span><br><span class="line"> ResourceMark rm;</span><br><span class="line"></span><br><span class="line"> <span class="built_in">g1_policy</span>()-><span class="built_in">note_gc_start</span>();</span><br><span class="line"></span><br><span class="line"> <span class="built_in">wait_for_root_region_scanning</span>();</span><br><span class="line"></span><br><span class="line"> <span class="built_in">print_heap_before_gc</span>();</span><br><span class="line"> <span class="built_in">print_heap_regions</span>();</span><br><span class="line"> <span class="built_in">trace_heap_before_gc</span>(_gc_tracer_stw);</span><br><span class="line"></span><br><span class="line"> _verifier-><span class="built_in">verify_region_sets_optional</span>();</span><br><span class="line"> _verifier-><span class="built_in">verify_dirty_young_regions</span>();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// We should not be doing initial mark unless the conc mark thread is running</span></span><br><span class="line"> <span class="keyword">if</span> (!_cm_thread-><span class="built_in">should_terminate</span>()) {</span><br><span class="line"> <span class="comment">// This call will decide whether this pause is an initial-mark</span></span><br><span class="line"> <span class="comment">// pause. If it is, in_initial_mark_gc() will return true</span></span><br><span class="line"> <span class="comment">// for the duration of this pause.</span></span><br><span class="line"> <span class="built_in">g1_policy</span>()-><span class="built_in">decide_on_conc_mark_initiation</span>();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// We do not allow initial-mark to be piggy-backed on a mixed GC.</span></span><br><span class="line"> <span class="built_in">assert</span>(!<span class="built_in">collector_state</span>()-><span class="built_in">in_initial_mark_gc</span>() ||</span><br><span class="line"> <span class="built_in">collector_state</span>()-><span class="built_in">in_young_only_phase</span>(), <span class="string">"sanity"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// We also do not allow mixed GCs during marking.</span></span><br><span class="line"> <span class="built_in">assert</span>(!<span class="built_in">collector_state</span>()-><span class="built_in">mark_or_rebuild_in_progress</span>() || <span class="built_in">collector_state</span>()-><span class="built_in">in_young_only_phase</span>(), <span class="string">"sanity"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Record whether this pause is an initial mark. When the current</span></span><br><span class="line"> <span class="comment">// thread has completed its logging output and it's safe to signal</span></span><br><span class="line"> <span class="comment">// the CM thread, the flag's value in the policy has been reset.</span></span><br><span class="line"> <span class="type">bool</span> should_start_conc_mark = <span class="built_in">collector_state</span>()-><span class="built_in">in_initial_mark_gc</span>();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Inner scope for scope based logging, timers, and stats collection</span></span><br><span class="line"> {</span><br><span class="line"> EvacuationInfo evacuation_info;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">collector_state</span>()-><span class="built_in">in_initial_mark_gc</span>()) {</span><br><span class="line"> <span class="comment">// We are about to start a marking cycle, so we increment the</span></span><br><span class="line"> <span class="comment">// full collection counter.</span></span><br><span class="line"> <span class="built_in">increment_old_marking_cycles_started</span>();</span><br><span class="line"> _cm-><span class="built_in">gc_tracer_cm</span>()-><span class="built_in">set_gc_cause</span>(<span class="built_in">gc_cause</span>());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> _gc_tracer_stw-><span class="built_in">report_yc_type</span>(<span class="built_in">collector_state</span>()-><span class="built_in">yc_type</span>());</span><br><span class="line"></span><br><span class="line"> GCTraceCPUTime tcpu;</span><br><span class="line"></span><br><span class="line"> G1HeapVerifier::G1VerifyType verify_type;</span><br><span class="line"> FormatBuffer<> <span class="built_in">gc_string</span>(<span class="string">"Pause Young "</span>);</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">collector_state</span>()-><span class="built_in">in_initial_mark_gc</span>()) {</span><br><span class="line"> gc_string.<span class="built_in">append</span>(<span class="string">"(Concurrent Start)"</span>);</span><br><span class="line"> verify_type = G1HeapVerifier::G1VerifyConcurrentStart;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (<span class="built_in">collector_state</span>()-><span class="built_in">in_young_only_phase</span>()) {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">collector_state</span>()-><span class="built_in">in_young_gc_before_mixed</span>()) {</span><br><span class="line"> gc_string.<span class="built_in">append</span>(<span class="string">"(Prepare Mixed)"</span>);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> gc_string.<span class="built_in">append</span>(<span class="string">"(Normal)"</span>);</span><br><span class="line"> }</span><br><span class="line"> verify_type = G1HeapVerifier::G1VerifyYoungNormal;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> gc_string.<span class="built_in">append</span>(<span class="string">"(Mixed)"</span>);</span><br><span class="line"> verify_type = G1HeapVerifier::G1VerifyMixed;</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">GCTraceTime</span>(Info, gc) <span class="built_in">tm</span>(gc_string, <span class="literal">NULL</span>, <span class="built_in">gc_cause</span>(), <span class="literal">true</span>);</span><br><span class="line"></span><br><span class="line"> uint active_workers = AdaptiveSizePolicy::<span class="built_in">calc_active_workers</span>(<span class="built_in">workers</span>()-><span class="built_in">total_workers</span>(),</span><br><span class="line"> <span class="built_in">workers</span>()-><span class="built_in">active_workers</span>(),</span><br><span class="line"> Threads::<span class="built_in">number_of_non_daemon_threads</span>());</span><br><span class="line"> active_workers = <span class="built_in">workers</span>()-><span class="built_in">update_active_workers</span>(active_workers);</span><br><span class="line"> <span class="built_in">log_info</span>(gc,task)(<span class="string">"Using %u workers of %u for evacuation"</span>, active_workers, <span class="built_in">workers</span>()-><span class="built_in">total_workers</span>());</span><br><span class="line"></span><br><span class="line"> <span class="function">TraceCollectorStats <span class="title">tcs</span><span class="params">(g1mm()->incremental_collection_counters())</span></span>;</span><br><span class="line"> <span class="function">TraceMemoryManagerStats <span class="title">tms</span><span class="params">(&_memory_manager, gc_cause(),</span></span></span><br><span class="line"><span class="params"><span class="function"> collector_state()->yc_type() == Mixed <span class="comment">/* allMemoryPoolsAffected */</span>)</span></span>;</span><br><span class="line"></span><br><span class="line"> <span class="function">G1HeapTransition <span class="title">heap_transition</span><span class="params">(<span class="keyword">this</span>)</span></span>;</span><br><span class="line"> <span class="type">size_t</span> heap_used_bytes_before_gc = <span class="built_in">used</span>();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Don't dynamically change the number of GC threads this early. A value of</span></span><br><span class="line"> <span class="comment">// 0 is used to indicate serial work. When parallel work is done,</span></span><br><span class="line"> <span class="comment">// it will be set.</span></span><br><span class="line"></span><br><span class="line"> { <span class="comment">// Call to jvmpi::post_class_unload_events must occur outside of active GC</span></span><br><span class="line"> IsGCActiveMark x;</span><br><span class="line"></span><br><span class="line"> <span class="built_in">gc_prologue</span>(<span class="literal">false</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (VerifyRememberedSets) {</span><br><span class="line"> <span class="built_in">log_info</span>(gc, verify)(<span class="string">"[Verifying RemSets before GC]"</span>);</span><br><span class="line"> VerifyRegionRemSetClosure v_cl;</span><br><span class="line"> <span class="built_in">heap_region_iterate</span>(&v_cl);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> _verifier-><span class="built_in">verify_before_gc</span>(verify_type);</span><br><span class="line"></span><br><span class="line"> _verifier-><span class="built_in">check_bitmaps</span>(<span class="string">"GC Start"</span>);</span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="keyword">if</span> COMPILER2_OR_JVMCI</span></span><br><span class="line"> DerivedPointerTable::<span class="built_in">clear</span>();</span><br><span class="line"><span class="meta">#<span class="keyword">endif</span></span></span><br><span class="line"></span><br><span class="line"> <span class="comment">// Please see comment in g1CollectedHeap.hpp and</span></span><br><span class="line"> <span class="comment">// G1CollectedHeap::ref_processing_init() to see how</span></span><br><span class="line"> <span class="comment">// reference processing currently works in G1.</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">// Enable discovery in the STW reference processor</span></span><br><span class="line"> _ref_processor_stw-><span class="built_in">enable_discovery</span>();</span><br><span class="line"></span><br><span class="line"> {</span><br><span class="line"> <span class="comment">// We want to temporarily turn off discovery by the</span></span><br><span class="line"> <span class="comment">// CM ref processor, if necessary, and turn it back on</span></span><br><span class="line"> <span class="comment">// on again later if we do. Using a scoped</span></span><br><span class="line"> <span class="comment">// NoRefDiscovery object will do this.</span></span><br><span class="line"> <span class="function">NoRefDiscovery <span class="title">no_cm_discovery</span><span class="params">(_ref_processor_cm)</span></span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Forget the current alloc region (we might even choose it to be part</span></span><br><span class="line"> <span class="comment">// of the collection set!).</span></span><br><span class="line"> _allocator-><span class="built_in">release_mutator_alloc_region</span>();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// This timing is only used by the ergonomics to handle our pause target.</span></span><br><span class="line"> <span class="comment">// It is unclear why this should not include the full pause. We will</span></span><br><span class="line"> <span class="comment">// investigate this in CR 7178365.</span></span><br><span class="line"> <span class="comment">//</span></span><br><span class="line"> <span class="comment">// Preserving the old comment here if that helps the investigation:</span></span><br><span class="line"> <span class="comment">//</span></span><br><span class="line"> <span class="comment">// The elapsed time induced by the start time below deliberately elides</span></span><br><span class="line"> <span class="comment">// the possible verification above.</span></span><br><span class="line"> <span class="type">double</span> sample_start_time_sec = os::<span class="built_in">elapsedTime</span>();</span><br><span class="line"></span><br><span class="line"> <span class="built_in">g1_policy</span>()-><span class="built_in">record_collection_pause_start</span>(sample_start_time_sec);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">collector_state</span>()-><span class="built_in">in_initial_mark_gc</span>()) {</span><br><span class="line"> <span class="built_in">concurrent_mark</span>()-><span class="built_in">pre_initial_mark</span>();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">g1_policy</span>()-><span class="built_in">finalize_collection_set</span>(target_pause_time_ms, &_survivor);</span><br><span class="line"></span><br><span class="line"> evacuation_info.<span class="built_in">set_collectionset_regions</span>(<span class="built_in">collection_set</span>()-><span class="built_in">region_length</span>());</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Make sure the remembered sets are up to date. This needs to be</span></span><br><span class="line"> <span class="comment">// done before register_humongous_regions_with_cset(), because the</span></span><br><span class="line"> <span class="comment">// remembered sets are used there to choose eager reclaim candidates.</span></span><br><span class="line"> <span class="comment">// If the remembered sets are not up to date we might miss some</span></span><br><span class="line"> <span class="comment">// entries that need to be handled.</span></span><br><span class="line"> <span class="built_in">g1_rem_set</span>()-><span class="built_in">cleanupHRRS</span>();</span><br><span class="line"></span><br><span class="line"> <span class="built_in">register_humongous_regions_with_cset</span>();</span><br><span class="line"></span><br><span class="line"> <span class="built_in">assert</span>(_verifier-><span class="built_in">check_cset_fast_test</span>(), <span class="string">"Inconsistency in the InCSetState table."</span>);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// We call this after finalize_cset() to</span></span><br><span class="line"> <span class="comment">// ensure that the CSet has been finalized.</span></span><br><span class="line"> _cm-><span class="built_in">verify_no_cset_oops</span>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (_hr_printer.<span class="built_in">is_active</span>()) {</span><br><span class="line"> <span class="function">G1PrintCollectionSetClosure <span class="title">cl</span><span class="params">(&_hr_printer)</span></span>;</span><br><span class="line"> _collection_set.<span class="built_in">iterate</span>(&cl);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Initialize the GC alloc regions.</span></span><br><span class="line"> _allocator-><span class="built_in">init_gc_alloc_regions</span>(evacuation_info);</span><br><span class="line"></span><br><span class="line"> <span class="function">G1ParScanThreadStateSet <span class="title">per_thread_states</span><span class="params">(<span class="keyword">this</span>, workers()->active_workers(), collection_set()->young_region_length())</span></span>;</span><br><span class="line"> <span class="built_in">pre_evacuate_collection_set</span>();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Actually do the work...</span></span><br><span class="line"> <span class="built_in">evacuate_collection_set</span>(&per_thread_states);</span><br><span class="line"></span><br><span class="line"> <span class="built_in">post_evacuate_collection_set</span>(evacuation_info, &per_thread_states);</span><br><span class="line"></span><br><span class="line"> <span class="type">const</span> <span class="type">size_t</span>* surviving_young_words = per_thread_states.<span class="built_in">surviving_young_words</span>();</span><br><span class="line"> <span class="built_in">free_collection_set</span>(&_collection_set, evacuation_info, surviving_young_words);</span><br><span class="line"></span><br><span class="line"> <span class="built_in">eagerly_reclaim_humongous_regions</span>();</span><br><span class="line"></span><br><span class="line"> <span class="built_in">record_obj_copy_mem_stats</span>();</span><br><span class="line"> _survivor_evac_stats.<span class="built_in">adjust_desired_plab_sz</span>();</span><br><span class="line"> _old_evac_stats.<span class="built_in">adjust_desired_plab_sz</span>();</span><br><span class="line"></span><br><span class="line"> <span class="type">double</span> start = os::<span class="built_in">elapsedTime</span>();</span><br><span class="line"> <span class="built_in">start_new_collection_set</span>();</span><br><span class="line"> <span class="built_in">g1_policy</span>()-><span class="built_in">phase_times</span>()-><span class="built_in">record_start_new_cset_time_ms</span>((os::<span class="built_in">elapsedTime</span>() - start) * <span class="number">1000.0</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">evacuation_failed</span>()) {</span><br><span class="line"> <span class="built_in">set_used</span>(<span class="built_in">recalculate_used</span>());</span><br><span class="line"> <span class="keyword">if</span> (_archive_allocator != <span class="literal">NULL</span>) {</span><br><span class="line"> _archive_allocator-><span class="built_in">clear_used</span>();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">for</span> (uint i = <span class="number">0</span>; i < ParallelGCThreads; i++) {</span><br><span class="line"> <span class="keyword">if</span> (_evacuation_failed_info_array[i].<span class="built_in">has_failed</span>()) {</span><br><span class="line"> _gc_tracer_stw-><span class="built_in">report_evacuation_failed</span>(_evacuation_failed_info_array[i]);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// The "used" of the the collection set have already been subtracted</span></span><br><span class="line"> <span class="comment">// when they were freed. Add in the bytes evacuated.</span></span><br><span class="line"> <span class="built_in">increase_used</span>(<span class="built_in">g1_policy</span>()-><span class="built_in">bytes_copied_during_gc</span>());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">collector_state</span>()-><span class="built_in">in_initial_mark_gc</span>()) {</span><br><span class="line"> <span class="comment">// We have to do this before we notify the CM threads that</span></span><br><span class="line"> <span class="comment">// they can start working to make sure that all the</span></span><br><span class="line"> <span class="comment">// appropriate initialization is done on the CM object.</span></span><br><span class="line"> <span class="built_in">concurrent_mark</span>()-><span class="built_in">post_initial_mark</span>();</span><br><span class="line"> <span class="comment">// Note that we don't actually trigger the CM thread at</span></span><br><span class="line"> <span class="comment">// this point. We do that later when we're sure that</span></span><br><span class="line"> <span class="comment">// the current thread has completed its logging output.</span></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">allocate_dummy_regions</span>();</span><br><span class="line"></span><br><span class="line"> _allocator-><span class="built_in">init_mutator_alloc_region</span>();</span><br><span class="line"></span><br><span class="line"> {</span><br><span class="line"> <span class="type">size_t</span> expand_bytes = _heap_sizing_policy-><span class="built_in">expansion_amount</span>();</span><br><span class="line"> <span class="keyword">if</span> (expand_bytes > <span class="number">0</span>) {</span><br><span class="line"> <span class="type">size_t</span> bytes_before = <span class="built_in">capacity</span>();</span><br><span class="line"> <span class="comment">// No need for an ergo logging here,</span></span><br><span class="line"> <span class="comment">// expansion_amount() does this when it returns a value > 0.</span></span><br><span class="line"> <span class="type">double</span> expand_ms;</span><br><span class="line"> <span class="keyword">if</span> (!<span class="built_in">expand</span>(expand_bytes, _workers, &expand_ms)) {</span><br><span class="line"> <span class="comment">// We failed to expand the heap. Cannot do anything about it.</span></span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">g1_policy</span>()-><span class="built_in">phase_times</span>()-><span class="built_in">record_expand_heap_time</span>(expand_ms);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// We redo the verification but now wrt to the new CSet which</span></span><br><span class="line"> <span class="comment">// has just got initialized after the previous CSet was freed.</span></span><br><span class="line"> _cm-><span class="built_in">verify_no_cset_oops</span>();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// This timing is only used by the ergonomics to handle our pause target.</span></span><br><span class="line"> <span class="comment">// It is unclear why this should not include the full pause. We will</span></span><br><span class="line"> <span class="comment">// investigate this in CR 7178365.</span></span><br><span class="line"> <span class="type">double</span> sample_end_time_sec = os::<span class="built_in">elapsedTime</span>();</span><br><span class="line"> <span class="type">double</span> pause_time_ms = (sample_end_time_sec - sample_start_time_sec) * MILLIUNITS;</span><br><span class="line"> <span class="type">size_t</span> total_cards_scanned = <span class="built_in">g1_policy</span>()-><span class="built_in">phase_times</span>()-><span class="built_in">sum_thread_work_items</span>(G1GCPhaseTimes::ScanRS, G1GCPhaseTimes::ScanRSScannedCards);</span><br><span class="line"> <span class="built_in">g1_policy</span>()-><span class="built_in">record_collection_pause_end</span>(pause_time_ms, total_cards_scanned, heap_used_bytes_before_gc);</span><br><span class="line"></span><br><span class="line"> evacuation_info.<span class="built_in">set_collectionset_used_before</span>(<span class="built_in">collection_set</span>()-><span class="built_in">bytes_used_before</span>());</span><br><span class="line"> evacuation_info.<span class="built_in">set_bytes_copied</span>(<span class="built_in">g1_policy</span>()-><span class="built_in">bytes_copied_during_gc</span>());</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (VerifyRememberedSets) {</span><br><span class="line"> <span class="built_in">log_info</span>(gc, verify)(<span class="string">"[Verifying RemSets after GC]"</span>);</span><br><span class="line"> VerifyRegionRemSetClosure v_cl;</span><br><span class="line"> <span class="built_in">heap_region_iterate</span>(&v_cl);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> _verifier-><span class="built_in">verify_after_gc</span>(verify_type);</span><br><span class="line"> _verifier-><span class="built_in">check_bitmaps</span>(<span class="string">"GC End"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="built_in">assert</span>(!_ref_processor_stw-><span class="built_in">discovery_enabled</span>(), <span class="string">"Postcondition"</span>);</span><br><span class="line"> _ref_processor_stw-><span class="built_in">verify_no_references_recorded</span>();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// CM reference discovery will be re-enabled if necessary.</span></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="keyword">ifdef</span> TRACESPINNING</span></span><br><span class="line"> ParallelTaskTerminator::<span class="built_in">print_termination_counts</span>();</span><br><span class="line"><span class="meta">#<span class="keyword">endif</span></span></span><br><span class="line"></span><br><span class="line"> <span class="built_in">gc_epilogue</span>(<span class="literal">false</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Print the remainder of the GC log output.</span></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">evacuation_failed</span>()) {</span><br><span class="line"> <span class="built_in">log_info</span>(gc)(<span class="string">"To-space exhausted"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">g1_policy</span>()-><span class="built_in">print_phases</span>();</span><br><span class="line"> heap_transition.<span class="built_in">print</span>();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// It is not yet to safe to tell the concurrent mark to</span></span><br><span class="line"> <span class="comment">// start as we have some optional output below. We don't want the</span></span><br><span class="line"> <span class="comment">// output from the concurrent mark thread interfering with this</span></span><br><span class="line"> <span class="comment">// logging output either.</span></span><br><span class="line"></span><br><span class="line"> _hrm.<span class="built_in">verify_optional</span>();</span><br><span class="line"> _verifier-><span class="built_in">verify_region_sets_optional</span>();</span><br><span class="line"></span><br><span class="line"> <span class="built_in">TASKQUEUE_STATS_ONLY</span>(<span class="built_in">print_taskqueue_stats</span>());</span><br><span class="line"> <span class="built_in">TASKQUEUE_STATS_ONLY</span>(<span class="built_in">reset_taskqueue_stats</span>());</span><br><span class="line"></span><br><span class="line"> <span class="built_in">print_heap_after_gc</span>();</span><br><span class="line"> <span class="built_in">print_heap_regions</span>();</span><br><span class="line"> <span class="built_in">trace_heap_after_gc</span>(_gc_tracer_stw);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// We must call G1MonitoringSupport::update_sizes() in the same scoping level</span></span><br><span class="line"> <span class="comment">// as an active TraceMemoryManagerStats object (i.e. before the destructor for the</span></span><br><span class="line"> <span class="comment">// TraceMemoryManagerStats is called) so that the G1 memory pools are updated</span></span><br><span class="line"> <span class="comment">// before any GC notifications are raised.</span></span><br><span class="line"> <span class="built_in">g1mm</span>()-><span class="built_in">update_sizes</span>();</span><br><span class="line"></span><br><span class="line"> _gc_tracer_stw-><span class="built_in">report_evacuation_info</span>(&evacuation_info);</span><br><span class="line"> _gc_tracer_stw-><span class="built_in">report_tenuring_threshold</span>(_g1_policy-><span class="built_in">tenuring_threshold</span>());</span><br><span class="line"> _gc_timer_stw-><span class="built_in">register_gc_end</span>();</span><br><span class="line"> _gc_tracer_stw-><span class="built_in">report_gc_end</span>(_gc_timer_stw-><span class="built_in">gc_end</span>(), _gc_timer_stw-><span class="built_in">time_partitions</span>());</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// It should now be safe to tell the concurrent mark thread to start</span></span><br><span class="line"> <span class="comment">// without its logging output interfering with the logging output</span></span><br><span class="line"> <span class="comment">// that came from the pause.</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (should_start_conc_mark) {</span><br><span class="line"> <span class="comment">// CAUTION: after the doConcurrentMark() call below,</span></span><br><span class="line"> <span class="comment">// the concurrent marking thread(s) could be running</span></span><br><span class="line"> <span class="comment">// concurrently with us. Make sure that anything after</span></span><br><span class="line"> <span class="comment">// this point does not assume that we are the only GC thread</span></span><br><span class="line"> <span class="comment">// running. Note: of course, the actual marking work will</span></span><br><span class="line"> <span class="comment">// not start until the safepoint itself is released in</span></span><br><span class="line"> <span class="comment">// SuspendibleThreadSet::desynchronize().</span></span><br><span class="line"> <span class="built_in">do_concurrent_mark</span>();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>往下走就是这一步<code>G1Policy::finalize_collection_set</code>,去处理新生代和老年代</p>
|
|
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="type">void</span> <span class="title">G1Policy::finalize_collection_set</span><span class="params">(<span class="type">double</span> target_pause_time_ms, G1SurvivorRegions* survivor)</span> </span>{</span><br><span class="line"> <span class="type">double</span> time_remaining_ms = _collection_set-><span class="built_in">finalize_young_part</span>(target_pause_time_ms, survivor);</span><br><span class="line"> _collection_set-><span class="built_in">finalize_old_part</span>(time_remaining_ms);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里分别调用了两个方法，可以看到剩余时间是往下传的，来看一下具体的方法</p>
|
|
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="type">double</span> <span class="title">G1CollectionSet::finalize_young_part</span><span class="params">(<span class="type">double</span> target_pause_time_ms, G1SurvivorRegions* survivors)</span> </span>{</span><br><span class="line"> <span class="type">double</span> young_start_time_sec = os::<span class="built_in">elapsedTime</span>();</span><br><span class="line"></span><br><span class="line"> <span class="built_in">finalize_incremental_building</span>();</span><br><span class="line"></span><br><span class="line"> <span class="built_in">guarantee</span>(target_pause_time_ms > <span class="number">0.0</span>,</span><br><span class="line"> <span class="string">"target_pause_time_ms = %1.6lf should be positive"</span>, target_pause_time_ms);</span><br><span class="line"></span><br><span class="line"> <span class="type">size_t</span> pending_cards = _policy-><span class="built_in">pending_cards</span>();</span><br><span class="line"> <span class="type">double</span> base_time_ms = _policy-><span class="built_in">predict_base_elapsed_time_ms</span>(pending_cards);</span><br><span class="line"> <span class="type">double</span> time_remaining_ms = <span class="built_in">MAX2</span>(target_pause_time_ms - base_time_ms, <span class="number">0.0</span>);</span><br><span class="line"></span><br><span class="line"> <span class="built_in">log_trace</span>(gc, ergo, cset)(<span class="string">"Start choosing CSet. pending cards: "</span> SIZE_FORMAT <span class="string">" predicted base time: %1.2fms remaining time: %1.2fms target pause time: %1.2fms"</span>,</span><br><span class="line"> pending_cards, base_time_ms, time_remaining_ms, target_pause_time_ms);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// The young list is laid with the survivor regions from the previous</span></span><br><span class="line"> <span class="comment">// pause are appended to the RHS of the young list, i.e.</span></span><br><span class="line"> <span class="comment">// [Newly Young Regions ++ Survivors from last pause].</span></span><br><span class="line"></span><br><span class="line"> uint survivor_region_length = survivors-><span class="built_in">length</span>();</span><br><span class="line"> uint eden_region_length = _g1h-><span class="built_in">eden_regions_count</span>();</span><br><span class="line"> <span class="built_in">init_region_lengths</span>(eden_region_length, survivor_region_length);</span><br><span class="line"></span><br><span class="line"> <span class="built_in">verify_young_cset_indices</span>();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Clear the fields that point to the survivor list - they are all young now.</span></span><br><span class="line"> survivors-><span class="built_in">convert_to_eden</span>();</span><br><span class="line"></span><br><span class="line"> _bytes_used_before = _inc_bytes_used_before;</span><br><span class="line"> time_remaining_ms = <span class="built_in">MAX2</span>(time_remaining_ms - _inc_predicted_elapsed_time_ms, <span class="number">0.0</span>);</span><br><span class="line"></span><br><span class="line"> <span class="built_in">log_trace</span>(gc, ergo, cset)(<span class="string">"Add young regions to CSet. eden: %u regions, survivors: %u regions, predicted young region time: %1.2fms, target pause time: %1.2fms"</span>,</span><br><span class="line"> eden_region_length, survivor_region_length, _inc_predicted_elapsed_time_ms, target_pause_time_ms);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// The number of recorded young regions is the incremental</span></span><br><span class="line"> <span class="comment">// collection set's current size</span></span><br><span class="line"> <span class="built_in">set_recorded_rs_lengths</span>(_inc_recorded_rs_lengths);</span><br><span class="line"></span><br><span class="line"> <span class="type">double</span> young_end_time_sec = os::<span class="built_in">elapsedTime</span>();</span><br><span class="line"> <span class="built_in">phase_times</span>()-><span class="built_in">record_young_cset_choice_time_ms</span>((young_end_time_sec - young_start_time_sec) * <span class="number">1000.0</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> time_remaining_ms;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>下面是老年代的部分</p>
|
|
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="type">void</span> <span class="title">G1CollectionSet::finalize_old_part</span><span class="params">(<span class="type">double</span> time_remaining_ms)</span> </span>{</span><br><span class="line"> <span class="type">double</span> non_young_start_time_sec = os::<span class="built_in">elapsedTime</span>();</span><br><span class="line"> <span class="type">double</span> predicted_old_time_ms = <span class="number">0.0</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">collector_state</span>()-><span class="built_in">in_mixed_phase</span>()) {</span><br><span class="line"> <span class="built_in">cset_chooser</span>()-><span class="built_in">verify</span>();</span><br><span class="line"> <span class="type">const</span> uint min_old_cset_length = _policy-><span class="built_in">calc_min_old_cset_length</span>();</span><br><span class="line"> <span class="type">const</span> uint max_old_cset_length = _policy-><span class="built_in">calc_max_old_cset_length</span>();</span><br><span class="line"></span><br><span class="line"> uint expensive_region_num = <span class="number">0</span>;</span><br><span class="line"> <span class="type">bool</span> check_time_remaining = _policy-><span class="built_in">adaptive_young_list_length</span>();</span><br><span class="line"></span><br><span class="line"> HeapRegion* hr = <span class="built_in">cset_chooser</span>()-><span class="built_in">peek</span>();</span><br><span class="line"> <span class="keyword">while</span> (hr != <span class="literal">NULL</span>) {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">old_region_length</span>() >= max_old_cset_length) {</span><br><span class="line"> <span class="comment">// Added maximum number of old regions to the CSet.</span></span><br><span class="line"> <span class="built_in">log_debug</span>(gc, ergo, cset)(<span class="string">"Finish adding old regions to CSet (old CSet region num reached max). old %u regions, max %u regions"</span>,</span><br><span class="line"> <span class="built_in">old_region_length</span>(), max_old_cset_length);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Stop adding regions if the remaining reclaimable space is</span></span><br><span class="line"> <span class="comment">// not above G1HeapWastePercent.</span></span><br><span class="line"> <span class="type">size_t</span> reclaimable_bytes = <span class="built_in">cset_chooser</span>()-><span class="built_in">remaining_reclaimable_bytes</span>();</span><br><span class="line"> <span class="type">double</span> reclaimable_percent = _policy-><span class="built_in">reclaimable_bytes_percent</span>(reclaimable_bytes);</span><br><span class="line"> <span class="type">double</span> threshold = (<span class="type">double</span>) G1HeapWastePercent;</span><br><span class="line"> <span class="keyword">if</span> (reclaimable_percent <= threshold) {</span><br><span class="line"> <span class="comment">// We've added enough old regions that the amount of uncollected</span></span><br><span class="line"> <span class="comment">// reclaimable space is at or below the waste threshold. Stop</span></span><br><span class="line"> <span class="comment">// adding old regions to the CSet.</span></span><br><span class="line"> <span class="built_in">log_debug</span>(gc, ergo, cset)(<span class="string">"Finish adding old regions to CSet (reclaimable percentage not over threshold). "</span></span><br><span class="line"> <span class="string">"old %u regions, max %u regions, reclaimable: "</span> SIZE_FORMAT <span class="string">"B (%1.2f%%) threshold: "</span> UINTX_FORMAT <span class="string">"%%"</span>,</span><br><span class="line"> <span class="built_in">old_region_length</span>(), max_old_cset_length, reclaimable_bytes, reclaimable_percent, G1HeapWastePercent);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">double</span> predicted_time_ms = <span class="built_in">predict_region_elapsed_time_ms</span>(hr);</span><br><span class="line"> <span class="keyword">if</span> (check_time_remaining) {</span><br><span class="line"> <span class="keyword">if</span> (predicted_time_ms > time_remaining_ms) {</span><br><span class="line"> <span class="comment">// Too expensive for the current CSet.</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">old_region_length</span>() >= min_old_cset_length) {</span><br><span class="line"> <span class="comment">// We have added the minimum number of old regions to the CSet,</span></span><br><span class="line"> <span class="comment">// we are done with this CSet.</span></span><br><span class="line"> <span class="built_in">log_debug</span>(gc, ergo, cset)(<span class="string">"Finish adding old regions to CSet (predicted time is too high). "</span></span><br><span class="line"> <span class="string">"predicted time: %1.2fms, remaining time: %1.2fms old %u regions, min %u regions"</span>,</span><br><span class="line"> predicted_time_ms, time_remaining_ms, <span class="built_in">old_region_length</span>(), min_old_cset_length);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// We'll add it anyway given that we haven't reached the</span></span><br><span class="line"> <span class="comment">// minimum number of old regions.</span></span><br><span class="line"> expensive_region_num += <span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">old_region_length</span>() >= min_old_cset_length) {</span><br><span class="line"> <span class="comment">// In the non-auto-tuning case, we'll finish adding regions</span></span><br><span class="line"> <span class="comment">// to the CSet if we reach the minimum.</span></span><br><span class="line"></span><br><span class="line"> <span class="built_in">log_debug</span>(gc, ergo, cset)(<span class="string">"Finish adding old regions to CSet (old CSet region num reached min). old %u regions, min %u regions"</span>,</span><br><span class="line"> <span class="built_in">old_region_length</span>(), min_old_cset_length);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// We will add this region to the CSet.</span></span><br><span class="line"> time_remaining_ms = <span class="built_in">MAX2</span>(time_remaining_ms - predicted_time_ms, <span class="number">0.0</span>);</span><br><span class="line"> predicted_old_time_ms += predicted_time_ms;</span><br><span class="line"> <span class="built_in">cset_chooser</span>()-><span class="built_in">pop</span>(); <span class="comment">// already have region via peek()</span></span><br><span class="line"> _g1h-><span class="built_in">old_set_remove</span>(hr);</span><br><span class="line"> <span class="built_in">add_old_region</span>(hr);</span><br><span class="line"></span><br><span class="line"> hr = <span class="built_in">cset_chooser</span>()-><span class="built_in">peek</span>();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (hr == <span class="literal">NULL</span>) {</span><br><span class="line"> <span class="built_in">log_debug</span>(gc, ergo, cset)(<span class="string">"Finish adding old regions to CSet (candidate old regions not available)"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (expensive_region_num > <span class="number">0</span>) {</span><br><span class="line"> <span class="comment">// We print the information once here at the end, predicated on</span></span><br><span class="line"> <span class="comment">// whether we added any apparently expensive regions or not, to</span></span><br><span class="line"> <span class="comment">// avoid generating output per region.</span></span><br><span class="line"> <span class="built_in">log_debug</span>(gc, ergo, cset)(<span class="string">"Added expensive regions to CSet (old CSet region num not reached min)."</span></span><br><span class="line"> <span class="string">"old: %u regions, expensive: %u regions, min: %u regions, remaining time: %1.2fms"</span>,</span><br><span class="line"> <span class="built_in">old_region_length</span>(), expensive_region_num, min_old_cset_length, time_remaining_ms);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">cset_chooser</span>()-><span class="built_in">verify</span>();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">stop_incremental_building</span>();</span><br><span class="line"></span><br><span class="line"> <span class="built_in">log_debug</span>(gc, ergo, cset)(<span class="string">"Finish choosing CSet. old: %u regions, predicted old region time: %1.2fms, time remaining: %1.2f"</span>,</span><br><span class="line"> <span class="built_in">old_region_length</span>(), predicted_old_time_ms, time_remaining_ms);</span><br><span class="line"></span><br><span class="line"> <span class="type">double</span> non_young_end_time_sec = os::<span class="built_in">elapsedTime</span>();</span><br><span class="line"> <span class="built_in">phase_times</span>()-><span class="built_in">record_non_young_cset_choice_time_ms</span>((non_young_end_time_sec - non_young_start_time_sec) * <span class="number">1000.0</span>);</span><br><span class="line"></span><br><span class="line"> QuickSort::<span class="built_in">sort</span>(_collection_set_regions, _collection_set_cur_length, compare_region_idx, <span class="literal">true</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>上面第三行是个判断，当前是否是 mixed 回收阶段，如果不是的话其实是没有老年代什么事的，所以可以看到代码基本是从这个 if 判断<br><code>if (collector_state()->in_mixed_phase()) {</code>开始往下走的<br>先写到这，偏向于做笔记用，有错轻拍</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>JVM</category>
|
|
<category>C++</category>
|
|
<category>GC</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>C++</tag>
|
|
<tag>JVM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Headscale渐入佳境补充篇-自定义中转derper使用反向代理</title>
|
|
<url>/2024/10/20/Headscale%E6%B8%90%E5%85%A5%E4%BD%B3%E5%A2%83%E8%A1%A5%E5%85%85%E7%AF%87-%E8%87%AA%E5%AE%9A%E4%B9%89%E4%B8%AD%E8%BD%ACderper%E4%BD%BF%E7%94%A8%E5%8F%8D%E5%90%91%E4%BB%A3%E7%90%86/</url>
|
|
<content><![CDATA[<p>之前在使用headscale的自建derper中转的时候，因为使用了acme管理的证书，虽然acme会自动续期，但是由于证书要做转换，没办法很方便的自动更新derper中映射的证书，因为最近在尝试迁移服务器，就在寻找是否有新的方法，正好就结合前面使用的<code>caddy</code>，只需要做好域名解析，<code>caddy</code> 就会自动加上<code>https</code>，那么在docker层面就可以不用额外增加证书啥的，各干各的事<br>docker还是一样的方式，差别就在于不用映射证书目录了</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">docker run --restart always --name derper -p 12345:12345 -p 3478:3478/udp -v /run/tailscale/tailscaled.sock:/var/run/tailscale/tailscaled.sock -e DERP_ADDR=:12345 -e DERP_DOMAIN=derper.domain.com -e DERP_VERIFY_CLIENTS=true -d dockerproxy.cn/yangchuansheng/derper:latest</span><br></pre></td></tr></table></figure>
|
|
<p><code>derper.domain.com</code> 需要换成自己的域名，端口也可以自己按需，只是还要注意下 <code>/run/tailscale/tailscaled.sock</code> 这个映射是为了让derper能够识别到这是同一个headscale下的客户端链接，防止被滥用，</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">curl -fsSL https://tailscale.com/install.sh | sh</span><br></pre></td></tr></table></figure>
|
|
<p>本机通过这个安装下，然后再登录headscale</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">tailscale up --login-server=http://headscaleip:headscaleport --accept-routes=true --accept-dns=false</span><br></pre></td></tr></table></figure>
|
|
<p>然后找到进程对应的sock文件，把它映射进去<br>接下去就是做反向代理，第一步先把域名解析做好，不然caddy没法做证书的，感觉caddy真的是懒人福音</p>
|
|
<figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="string">derper.domain.com</span> {</span><br><span class="line"> <span class="comment"># 反向代理的地址</span></span><br><span class="line"> <span class="string">reverse_proxy</span> <span class="number">127.0</span><span class="number">.0</span><span class="number">.1</span><span class="string">:12345</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>就这几行，是不是超方便，这样就省去了隔段时间要去转换下证书，重启derper的麻烦了，虽然也可以自动化，但是作为一个shell不那么熟练的，又怕权限控制不好，重要目录被删掉的，还是这样比较省力<br><img data-src="https://img.nicksxs.me/blog/1Q1Sig.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>headscale</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>headscale</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Java 线程池系列-准备篇</title>
|
|
<url>/2024/02/04/Java-%E7%BA%BF%E7%A8%8B%E6%B1%A0%E7%B3%BB%E5%88%97-%E5%87%86%E5%A4%87%E7%AF%87/</url>
|
|
<content><![CDATA[<p>Java 线程池是 Java 并发体系里的重要组成部分，不过说起线程池一般都是几个参数，以及参数的含义和逻辑，为了更适合刚开始学习的同学或者说有一些会对这么个习惯性的概括讲法有点疑惑的，我们就从零开始，从最基础的开始看起，先来看下里面用来判断线程池状态和线程数量的部分</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">final</span> <span class="type">AtomicInteger</span> <span class="variable">ctl</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">AtomicInteger</span>(ctlOf(RUNNING, <span class="number">0</span>));</span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">COUNT_BITS</span> <span class="operator">=</span> Integer.SIZE - <span class="number">3</span>;</span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">CAPACITY</span> <span class="operator">=</span> (<span class="number">1</span> << COUNT_BITS) - <span class="number">1</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">// runState is stored in the high-order bits</span></span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">RUNNING</span> <span class="operator">=</span> -<span class="number">1</span> << COUNT_BITS;</span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">SHUTDOWN</span> <span class="operator">=</span> <span class="number">0</span> << COUNT_BITS;</span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">STOP</span> <span class="operator">=</span> <span class="number">1</span> << COUNT_BITS;</span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">TIDYING</span> <span class="operator">=</span> <span class="number">2</span> << COUNT_BITS;</span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">TERMINATED</span> <span class="operator">=</span> <span class="number">3</span> << COUNT_BITS;</span><br><span class="line"></span><br><span class="line"><span class="comment">// Packing and unpacking ctl</span></span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="type">int</span> <span class="title function_">runStateOf</span><span class="params">(<span class="type">int</span> c)</span> { <span class="keyword">return</span> c & ~CAPACITY; }</span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="type">int</span> <span class="title function_">workerCountOf</span><span class="params">(<span class="type">int</span> c)</span> { <span class="keyword">return</span> c & CAPACITY; }</span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="type">int</span> <span class="title function_">ctlOf</span><span class="params">(<span class="type">int</span> rs, <span class="type">int</span> wc)</span> { <span class="keyword">return</span> rs | wc; }</span><br></pre></td></tr></table></figure>
|
|
<p>这里我们先看下 <code>COUNT_BITS</code> 它是 <code>Integer.SIZE - 3</code>，<code>Integer.SIZE</code> 这个是 32，也就是代表了 Java 用了四个 <code>Byte</code> 来存一个 <code>Integer</code>, 那么 <code>COUNT_BITS</code> 就是 29，<br>这个 29 是用了干嘛的呢，逐步往下看，为什么先介绍 <code>COUNT_BITS</code>，因为第一行的 <code>ctl</code> 的值是通过 <code>ctlOf(RUNNING, 0)</code> 算出来的，而 <code>RUNNING</code> 又是通过等于 <code>-1</code> 左移 <code>COUNT_BITS</code> 位来得到，移动 29 位后，还剩下三位，而这三位只需要表示后面五种状态，其中只有 <code>RUNNING</code> 是负值，这样就巧妙的只利用了 ctl 的前三位来存储线程池的状态而后面的 29 位就是用来存储线程数量的，具体怎么取呢，我们在来看下 <code>CAPACITY</code> 它是 <code>(1 << COUNT_BITS) - 1</code> 把 1 左移 29 位后减 1，也就是 536870911，这里再补充下，左移一位就等于是乘以 2，左移 29 位就是乘以 2 的 29 次，减了 1 就是 29 位能存的最大值，代表线程池最大的线程容量，用二进制来表示就是 32 位长度的int，前三位是 0，后 29 位是 1，代表了线程池的容量，那么 <code>runStateOf</code> 和 <code>workerCountOf</code> 的理解就很简单了，<br><code>runStateOf</code> 是 c 跟 CAPACITY 取反之后的值做”与”操作，<code>CAPACITY</code> 取反就变成了前三位是 1，后面的二十九位是 0，而对于”与”操作，后面的二十九位无论是什么都会被忽略，也就是取了前三位的值<br><code>workerCountOf</code> 正好相反，直接跟 <code>CAPACITY</code> 做 “与” 操作，也就是取后二十九位的值<br><code>ctlOf</code> 就是直接对 <code>rs</code> 与 <code>wc</code> 做了 “或” 操作<br>而前面也提到了只有 <code>RUNNING</code> 是负数，那么就可以直接做大小比较来判断线程池状态<br>这个可能对很多同学来说是非常简单的，但是对于一部分同学来说就是个拦路虎，希望能对这部分同学有所帮助。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Java 线程池系列-第三篇</title>
|
|
<url>/2024/02/24/Java-%E7%BA%BF%E7%A8%8B%E6%B1%A0%E7%B3%BB%E5%88%97-%E7%AC%AC%E4%B8%89%E7%AF%87/</url>
|
|
<content><![CDATA[<p>第三篇我们要来讲下 Worker 这个“打工人”，线程池里实际在干活的同学<br>Worker 实现了 Runnable 接口，所以在前面介绍的 <code>addWorker</code> 中<br>线程启动其实就调用了下面的 run 方法，而 run 方法则调用了内部的<br><code>runWorker</code> 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"></span><br><span class="line"><span class="keyword">private</span> <span class="keyword">final</span> <span class="keyword">class</span> <span class="title class_">Worker</span></span><br><span class="line"> <span class="keyword">extends</span> <span class="title class_">AbstractQueuedSynchronizer</span></span><br><span class="line"> <span class="keyword">implements</span> <span class="title class_">Runnable</span></span><br><span class="line">{</span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * This class will never be serialized, but we provide a</span></span><br><span class="line"><span class="comment"> * serialVersionUID to suppress a javac warning.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">long</span> <span class="variable">serialVersionUID</span> <span class="operator">=</span> <span class="number">6138294804551838833L</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/** Thread this worker is running in. Null if factory fails. */</span></span><br><span class="line"> <span class="keyword">final</span> Thread thread;</span><br><span class="line"> <span class="comment">/** Initial task to run. Possibly null. */</span></span><br><span class="line"> Runnable firstTask;</span><br><span class="line"> <span class="comment">/** Per-thread task counter */</span></span><br><span class="line"> <span class="keyword">volatile</span> <span class="type">long</span> completedTasks;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Creates with given first task and thread from ThreadFactory.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> firstTask the first task (null if none)</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> Worker(Runnable firstTask) {</span><br><span class="line"> setState(-<span class="number">1</span>); <span class="comment">// inhibit interrupts until runWorker</span></span><br><span class="line"> <span class="built_in">this</span>.firstTask = firstTask;</span><br><span class="line"> <span class="built_in">this</span>.thread = getThreadFactory().newThread(<span class="built_in">this</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/** Delegates main run loop to outer runWorker */</span></span><br><span class="line"> <span class="comment">// 启动线程 Worker</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> runWorker(<span class="built_in">this</span>);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>介绍下这个 <code>runWorker</code> 方法<br>这里就会先处理 firstTask，如果 firstTask 是 null 则会去队列里取<br>里面会有两个钩子，一个是执行前，一个是执行后，注意这个循环是没有常规<br>退出逻辑的，说明是一直在往队列里获取，除非获取队列里的任务超时失败了<br>这样就到最后会去执行 <code>processWorkerExit</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">final</span> <span class="keyword">void</span> <span class="title function_">runWorker</span><span class="params">(Worker w)</span> {</span><br><span class="line"> <span class="type">Thread</span> <span class="variable">wt</span> <span class="operator">=</span> Thread.currentThread();</span><br><span class="line"> <span class="type">Runnable</span> <span class="variable">task</span> <span class="operator">=</span> w.firstTask;</span><br><span class="line"> w.firstTask = <span class="literal">null</span>;</span><br><span class="line"> w.unlock(); <span class="comment">// allow interrupts</span></span><br><span class="line"> <span class="type">boolean</span> <span class="variable">completedAbruptly</span> <span class="operator">=</span> <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// firstTask 不为空或者从队列里获取到任务</span></span><br><span class="line"> <span class="keyword">while</span> (task != <span class="literal">null</span> || (task = getTask()) != <span class="literal">null</span>) {</span><br><span class="line"> w.lock();</span><br><span class="line"> <span class="comment">// If pool is stopping, ensure thread is interrupted;</span></span><br><span class="line"> <span class="comment">// if not, ensure thread is not interrupted. This</span></span><br><span class="line"> <span class="comment">// requires a recheck in second case to deal with</span></span><br><span class="line"> <span class="comment">// shutdownNow race while clearing interrupt</span></span><br><span class="line"> <span class="keyword">if</span> ((runStateAtLeast(ctl.get(), STOP) ||</span><br><span class="line"> (Thread.interrupted() &&</span><br><span class="line"> runStateAtLeast(ctl.get(), STOP))) &&</span><br><span class="line"> !wt.isInterrupted())</span><br><span class="line"> wt.interrupt();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 前置执行钩子</span></span><br><span class="line"> beforeExecute(wt, task);</span><br><span class="line"> <span class="type">Throwable</span> <span class="variable">thrown</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 任务执行</span></span><br><span class="line"> task.run();</span><br><span class="line"> } <span class="keyword">catch</span> (RuntimeException x) {</span><br><span class="line"> thrown = x; <span class="keyword">throw</span> x;</span><br><span class="line"> } <span class="keyword">catch</span> (Error x) {</span><br><span class="line"> thrown = x; <span class="keyword">throw</span> x;</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable x) {</span><br><span class="line"> thrown = x; <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">Error</span>(x);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="comment">// 执行完后的钩子</span></span><br><span class="line"> afterExecute(task, thrown);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="comment">// 最后会执行到这，清空任务</span></span><br><span class="line"> task = <span class="literal">null</span>;</span><br><span class="line"> w.completedTasks++;</span><br><span class="line"> w.unlock();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> completedAbruptly = <span class="literal">false</span>;</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="comment">// 循环的最后就会去退出Worker</span></span><br><span class="line"> processWorkerExit(w, completedAbruptly);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>而这个 <code>processWorkerExit</code> 也有个比较特殊的逻辑<br>照理我们会想到这里可能就是线程在处理完队列里的任务<br>以后，就会判断下是否要退出，如果是核心线程的话就不用<br>如果是非核心线程就会退出了，但是这里其实有个替换逻辑<br>就是最后有个 addWorker 调用</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">processWorkerExit</span><span class="params">(Worker w, <span class="type">boolean</span> completedAbruptly)</span> {</span><br><span class="line"> <span class="keyword">if</span> (completedAbruptly) <span class="comment">// If abrupt, then workerCount wasn't adjusted</span></span><br><span class="line"> decrementWorkerCount();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">final</span> <span class="type">ReentrantLock</span> <span class="variable">mainLock</span> <span class="operator">=</span> <span class="built_in">this</span>.mainLock;</span><br><span class="line"> mainLock.lock();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> completedTaskCount += w.completedTasks;</span><br><span class="line"> workers.remove(w);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> mainLock.unlock();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> tryTerminate();</span><br><span class="line"></span><br><span class="line"> <span class="type">int</span> <span class="variable">c</span> <span class="operator">=</span> ctl.get();</span><br><span class="line"> <span class="comment">// 判断状态</span></span><br><span class="line"> <span class="keyword">if</span> (runStateLessThan(c, STOP)) {</span><br><span class="line"> <span class="keyword">if</span> (!completedAbruptly) {</span><br><span class="line"> <span class="comment">// 核心线程是否也需要退出</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">min</span> <span class="operator">=</span> allowCoreThreadTimeOut ? <span class="number">0</span> : corePoolSize;</span><br><span class="line"> <span class="keyword">if</span> (min == <span class="number">0</span> && ! workQueue.isEmpty())</span><br><span class="line"> min = <span class="number">1</span>;</span><br><span class="line"> <span class="comment">// 这里代表非核心线程就可以不用替换，也就是回收线程了</span></span><br><span class="line"> <span class="keyword">if</span> (workerCountOf(c) >= min)</span><br><span class="line"> <span class="keyword">return</span>; <span class="comment">// replacement not needed</span></span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 否则是替换线程</span></span><br><span class="line"> addWorker(<span class="literal">null</span>, <span class="literal">false</span>);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里主要讲了线程 Worker 运行的逻辑</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Java 线程池系列-第四篇</title>
|
|
<url>/2024/03/03/Java-%E7%BA%BF%E7%A8%8B%E6%B1%A0%E7%B3%BB%E5%88%97-%E7%AC%AC%E5%9B%9B%E7%AF%87/</url>
|
|
<content><![CDATA[<p>这一篇主要补充两个内容，第一部分就是获取任务的逻辑<br>首先是状态判断，如果是停止了，SHUTDOWN或更大的了，就需要减小工作线程数量<br>并返回 null，使得工作线程 worker 退出，然后再判断线程数量和超时，同样如果超过了就会返回 null<br>然后就是去阻塞队列里获取任务，这里是阻塞着获取的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> Runnable <span class="title function_">getTask</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">timedOut</span> <span class="operator">=</span> <span class="literal">false</span>; <span class="comment">// Did the last poll() time out?</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (;;) {</span><br><span class="line"> <span class="type">int</span> <span class="variable">c</span> <span class="operator">=</span> ctl.get();</span><br><span class="line"> <span class="type">int</span> <span class="variable">rs</span> <span class="operator">=</span> runStateOf(c);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Check if queue empty only if necessary.</span></span><br><span class="line"> <span class="comment">// 状态异常，如果是大于等于 SHUTDOWN 的，则协助关闭线程池</span></span><br><span class="line"> <span class="keyword">if</span> (rs >= SHUTDOWN && (rs >= STOP || workQueue.isEmpty())) {</span><br><span class="line"> decrementWorkerCount();</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">int</span> <span class="variable">wc</span> <span class="operator">=</span> workerCountOf(c);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Are workers subject to culling?</span></span><br><span class="line"> <span class="type">boolean</span> <span class="variable">timed</span> <span class="operator">=</span> allowCoreThreadTimeOut || wc > corePoolSize;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 如果线程数量超过核心线程数，则帮助减少线程</span></span><br><span class="line"> <span class="keyword">if</span> ((wc > maximumPoolSize || (timed && timedOut))</span><br><span class="line"> && (wc > <span class="number">1</span> || workQueue.isEmpty())) {</span><br><span class="line"> <span class="keyword">if</span> (compareAndDecrementWorkerCount(c))</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 如果前面的不符合，则从阻塞队列获取任务</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">Runnable</span> <span class="variable">r</span> <span class="operator">=</span> timed ?</span><br><span class="line"> workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) :</span><br><span class="line"> workQueue.take();</span><br><span class="line"> <span class="keyword">if</span> (r != <span class="literal">null</span>)</span><br><span class="line"> <span class="keyword">return</span> r;</span><br><span class="line"> timedOut = <span class="literal">true</span>;</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException retry) {</span><br><span class="line"> timedOut = <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>阻塞队列的 poll 主要是通过锁，和notEmpty这个 condition 来等待制定的时间<br>指定时间后开始 dequeue 出队</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> E <span class="title function_">poll</span><span class="params">(<span class="type">long</span> timeout, TimeUnit unit)</span> <span class="keyword">throws</span> InterruptedException {</span><br><span class="line"> <span class="type">long</span> <span class="variable">nanos</span> <span class="operator">=</span> unit.toNanos(timeout);</span><br><span class="line"> <span class="keyword">final</span> <span class="type">ReentrantLock</span> <span class="variable">lock</span> <span class="operator">=</span> <span class="built_in">this</span>.lock;</span><br><span class="line"> lock.lockInterruptibly();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">while</span> (count == <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">if</span> (nanos <= <span class="number">0</span>)</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> nanos = notEmpty.awaitNanos(nanos);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> dequeue();</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> lock.unlock();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>第二部分比较重要，有的同学的问题是为什么不一开始开到最大线程，而是达到核心线程数后就进队列了，<br>其实池化技术最简单的原因就是希望能够复用这些线程，因为创建销毁他们的成本太大了，如果直接最大线程数的话<br>其实都不用用到线程池技术了，直接有多少任务就开多少线程，用完就丢了，阐述下我认为比较重要的概念点</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 021 合并两个有序链表 ( Merge Two Sorted Lists ) 题解分析</title>
|
|
<url>/2021/10/07/Leetcode-021-%E5%90%88%E5%B9%B6%E4%B8%A4%E4%B8%AA%E6%9C%89%E5%BA%8F%E9%93%BE%E8%A1%A8-Merge-Two-Sorted-Lists-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h3 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h3><p>Merge two sorted linked lists and return it as a sorted list. The list should be made by splicing together the nodes of the first two lists.</p>
|
|
<p>将两个升序链表合并为一个新的 升序 链表并返回。新链表是通过拼接给定的两个链表的所有节点组成的。 </p>
|
|
<h4 id="示例-1"><a href="#示例-1" class="headerlink" title="示例 1"></a>示例 1</h4><p><img data-src="https://img.nicksxs.me/uPic/R1wHlz.jpg"></p>
|
|
<blockquote>
|
|
<p><strong>输入</strong>：l1 = [1,2,4], l2 = [1,3,4]<br><strong>输出</strong>：[1,1,2,3,4,4]</p>
|
|
</blockquote>
|
|
<h4 id="示例-2"><a href="#示例-2" class="headerlink" title="示例 2"></a>示例 2</h4><blockquote>
|
|
<p><strong>输入</strong>: l1 = [], l2 = []<br><strong>输出</strong>: []</p>
|
|
</blockquote>
|
|
<h4 id="示例-3"><a href="#示例-3" class="headerlink" title="示例 3"></a>示例 3</h4><blockquote>
|
|
<p><strong>输入</strong>: l1 = [], l2 = [0]<br><strong>输出</strong>: [0]</p>
|
|
</blockquote>
|
|
<h3 id="简要分析"><a href="#简要分析" class="headerlink" title="简要分析"></a>简要分析</h3><p>这题是 Easy 的，看着也挺简单，两个链表进行合并，就是比较下大小，可能将就点的话最好就在两个链表中原地合并</p>
|
|
<h3 id="题解代码"><a href="#题解代码" class="headerlink" title="题解代码"></a>题解代码</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> ListNode <span class="title function_">mergeTwoLists</span><span class="params">(ListNode l1, ListNode l2)</span> {</span><br><span class="line"> <span class="comment">// 下面两个if判断了入参的边界，如果其一为null，直接返回另一个就可以了</span></span><br><span class="line"> <span class="keyword">if</span> (l1 == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> l2;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (l2 == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> l1;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// new 一个合并后的头结点</span></span><br><span class="line"> <span class="type">ListNode</span> <span class="variable">merged</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ListNode</span>();</span><br><span class="line"> <span class="comment">// 这个是当前节点</span></span><br><span class="line"> <span class="type">ListNode</span> <span class="variable">current</span> <span class="operator">=</span> merged;</span><br><span class="line"> <span class="comment">// 一开始给这个while加了l1和l2不全为null的条件，后面想了下不需要</span></span><br><span class="line"> <span class="comment">// 因为内部前两个if就是跳出条件</span></span><br><span class="line"> <span class="keyword">while</span> (<span class="literal">true</span>) {</span><br><span class="line"> <span class="keyword">if</span> (l1 == <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// 这里其实跟开头类似，只不过这里需要将l2剩余部分接到merged链表后面</span></span><br><span class="line"> <span class="comment">// 所以不能是直接current = l2，这样就是把后面的直接丢了</span></span><br><span class="line"> current.val = l2.val;</span><br><span class="line"> current.next = l2.next;</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (l2 == <span class="literal">null</span>) {</span><br><span class="line"> current.val = l1.val;</span><br><span class="line"> current.next = l1.next;</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 这里是两个链表都不为空的时候，就比较下大小</span></span><br><span class="line"> <span class="keyword">if</span> (l1.val < l2.val) {</span><br><span class="line"> current.val = l1.val;</span><br><span class="line"> l1 = l1.next;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> current.val = l2.val;</span><br><span class="line"> l2 = l2.next;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 这里是new个新的，其实也可以放在循环头上</span></span><br><span class="line"> current.next = <span class="keyword">new</span> <span class="title class_">ListNode</span>();</span><br><span class="line"> current = current.next;</span><br><span class="line"> }</span><br><span class="line"> current = <span class="literal">null</span>;</span><br><span class="line"> <span class="comment">// 返回这个头结点</span></span><br><span class="line"> <span class="keyword">return</span> merged;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<h3 id="结果"><a href="#结果" class="headerlink" title="结果"></a>结果</h3><p><img data-src="https://img.nicksxs.me/uPic/4iHpzc.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Java 线程池系列-实战篇</title>
|
|
<url>/2024/03/17/Java-%E7%BA%BF%E7%A8%8B%E6%B1%A0%E7%B3%BB%E5%88%97-%E5%AE%9E%E6%88%98%E7%AF%87/</url>
|
|
<content><![CDATA[<p>线程池在实际使用过程中，有时候在理解比较偏理论的时候会出现一些判断错误，这里我们就来看一个实际的案例</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">ThreadPoolExecutor</span> <span class="variable">threadPoolExecutor</span> <span class="operator">=</span></span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">ThreadPoolExecutor</span>(<span class="number">1</span>, <span class="number">1</span>,</span><br><span class="line"> <span class="number">0</span>, TimeUnit.MINUTES,</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">MyArrayBlockingQueue</span><>(<span class="number">2</span>));</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> <span class="keyword">throws</span> InterruptedException {</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < <span class="number">100</span>; i++) {</span><br><span class="line"> Thread.sleep(<span class="number">100</span>);</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">j</span> <span class="operator">=</span> <span class="number">0</span>; j < <span class="number">3</span>; j++) {</span><br><span class="line"> threadPoolExecutor.execute(() -> {</span><br><span class="line"> <span class="type">int</span> <span class="variable">a</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> });</span><br><span class="line"> }</span><br><span class="line"> System.out.println(<span class="string">"===============> 详情任务 - 任务处理完成"</span>);</span><br><span class="line"> }</span><br><span class="line"> System.out.println(<span class="string">"都执行完成了"</span>);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p><code>MyArrayBlockingQueue</code> 是我复制的 <code>ArrayBlockingQueue</code> 加了点日志，可以认为就是一样的，这种情况下<br>执行过程是怎么样的呢， 队列长度是 2，核心线程数和最大线程数都是 1，提交任务是采用了两层循环，内层是循环三次，往线程池里提交任务，然后内层循环完了以后会重新睡眠 100 毫秒<br>在进入下一次外层循环，如果能一眼看出来问题的说明对线程池了解得很深入了，如果没有的话我们就一起来看下<br>先说下结论，这个代码会出现拒绝异常<br><img data-src="https://img.nicksxs.me/blog/9eIEXq.png"><br>考虑下是什么原因呢，是不是我线程数太少了，放大一些，感觉符合直觉一点<br>修改成</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">ThreadPoolExecutor</span> <span class="variable">threadPoolExecutor</span> <span class="operator">=</span></span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">ThreadPoolExecutor</span>(<span class="number">100</span>, <span class="number">100</span>,</span><br><span class="line"> <span class="number">0</span>, TimeUnit.MINUTES,</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">MyArrayBlockingQueue</span><>(<span class="number">2</span>));</span><br></pre></td></tr></table></figure>
|
|
<p>然而还是一样<br><img data-src="https://img.nicksxs.me/blog/cezQty.png"><br>只不过晚了点出现，那么问题出在哪呢<br>为什么我要去重写这个 <code>MyArrayBlockingQueue</code>，就是为了找到原因，其实很多讲解线程池的都是讲了线程池的参数，什么队列是链表的，数组的<br>但是没有讲到我是怎么往队列塞任务，怎么从队列取任务的呢</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">offer</span><span class="params">(E e)</span> {</span><br><span class="line"> checkNotNull(e);</span><br><span class="line"> <span class="keyword">final</span> <span class="type">ReentrantLock</span> <span class="variable">lock</span> <span class="operator">=</span> <span class="built_in">this</span>.lock;</span><br><span class="line"> lock.lock();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (count == items.length) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> {</span><br><span class="line"> enqueue(e);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> lock.unlock();</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里是往队列里塞任务，注意这里需要获得锁，<br>而对于获取任务呢</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> E <span class="title function_">take</span><span class="params">()</span> <span class="keyword">throws</span> InterruptedException {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">ReentrantLock</span> <span class="variable">lock</span> <span class="operator">=</span> <span class="built_in">this</span>.lock;</span><br><span class="line"> lock.lockInterruptibly();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">while</span> (count == <span class="number">0</span>)</span><br><span class="line"> notEmpty.await();</span><br><span class="line"> <span class="keyword">return</span> dequeue();</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> lock.unlock();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>注意这里也需要获得锁，当我一个线程池的线程数进入稳定状态，也就是保持一定数量的线程不变的情况下<br>上面是一种比较可能的情况，即核心线程数等于最大线程数，那么我在提交任务的时候是非常快的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">if</span> (command == <span class="literal">null</span>)</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">NullPointerException</span>();</span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> * Proceed in 3 steps:</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * 1. If fewer than corePoolSize threads are running, try to</span></span><br><span class="line"><span class="comment"> * start a new thread with the given command as its first</span></span><br><span class="line"><span class="comment"> * task. The call to addWorker atomically checks runState and</span></span><br><span class="line"><span class="comment"> * workerCount, and so prevents false alarms that would add</span></span><br><span class="line"><span class="comment"> * threads when it shouldn't, by returning false.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * 2. If a task can be successfully queued, then we still need</span></span><br><span class="line"><span class="comment"> * to double-check whether we should have added a thread</span></span><br><span class="line"><span class="comment"> * (because existing ones died since last checking) or that</span></span><br><span class="line"><span class="comment"> * the pool shut down since entry into this method. So we</span></span><br><span class="line"><span class="comment"> * recheck state and if necessary roll back the enqueuing if</span></span><br><span class="line"><span class="comment"> * stopped, or start a new thread if there are none.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * 3. If we cannot queue task, then we try to add a new</span></span><br><span class="line"><span class="comment"> * thread. If it fails, we know we are shut down or saturated</span></span><br><span class="line"><span class="comment"> * and so reject the task.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">c</span> <span class="operator">=</span> ctl.get();</span><br><span class="line"> <span class="keyword">if</span> (workerCountOf(c) < corePoolSize) {</span><br><span class="line"> <span class="keyword">if</span> (addWorker(command, <span class="literal">true</span>))</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> c = ctl.get();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (isRunning(c) && workQueue.offer(command)) {</span><br><span class="line"> <span class="type">int</span> <span class="variable">recheck</span> <span class="operator">=</span> ctl.get();</span><br><span class="line"> <span class="keyword">if</span> (! isRunning(recheck) && remove(command))</span><br><span class="line"> reject(command);</span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span> (workerCountOf(recheck) == <span class="number">0</span>)</span><br><span class="line"> addWorker(<span class="literal">null</span>, <span class="literal">false</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span> (!addWorker(command, <span class="literal">false</span>))</span><br><span class="line"> reject(command);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>再来看下这段代码，第一步只需要判断是否为空，第二步就是判断核心线程数量，明显我说的情况，前面两步就直接过去了<br>然后就是判断线程池运行状态和往队列里塞任务了，但是线程运行完一个任务主动从队列里获取则需要更多的逻辑<br>这样就造成了我往队列里塞任务会比获取任务快很多，队列一满，就会抛出拒绝异常<br>即使我把线程数量放大到 100 还是一样，只不过会出现的慢一点，那么口说无凭，我们来验证下，提交任务过快，那么我在提交<br>方法里做个延迟</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">offer</span><span class="params">(E e)</span> {</span><br><span class="line"> checkNotNull(e);</span><br><span class="line"> <span class="keyword">final</span> <span class="type">ReentrantLock</span> <span class="variable">lock</span> <span class="operator">=</span> <span class="built_in">this</span>.lock;</span><br><span class="line"> lock.lock();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (count == items.length) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> {</span><br><span class="line"> enqueue(e);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> lock.unlock();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> Thread.sleep(<span class="number">1000</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException ex) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">RuntimeException</span>(ex);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这样就没啥问题了<br><img data-src="https://img.nicksxs.me/blog/S7eRgP.png"></p>
|
|
<p>除了最后这个加延时，其他的直接用 <code>ArrayBlockingQueue</code> 就可以实验，实操一下会对这个逻辑有更深的理解</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 028 实现 strStr() ( Implement strStr() ) 题解分析</title>
|
|
<url>/2021/10/31/Leetcode-028-%E5%AE%9E%E7%8E%B0-strStr-Implement-strStr-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h3 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h3><p>Implement <code>strStr()</code>.</p>
|
|
<p>Return the index of the first occurrence of needle in haystack, or <code>-1</code> if <code>needle</code> is not part of <code>haystack</code>.</p>
|
|
<h4 id="Clarification"><a href="#Clarification" class="headerlink" title="Clarification:"></a>Clarification:</h4><p>What should we return when <code>needle</code> is an empty string? This is a great question to ask during an interview.</p>
|
|
<p>For the purpose of this problem, we will return 0 when <code>needle</code> is an empty string. This is consistent to C’s <code>strstr()</code> and Java’s <code>indexOf()</code>.</p>
|
|
<h3 id="示例"><a href="#示例" class="headerlink" title="示例"></a>示例</h3><p>Example 1:</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Input: haystack = "hello", needle = "ll"</span><br><span class="line">Output: 2</span><br></pre></td></tr></table></figure>
|
|
<p>Example 2:</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Input: haystack = "aaaaa", needle = "bba"</span><br><span class="line">Output: -1</span><br></pre></td></tr></table></figure>
|
|
<p>Example 3:</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Input: haystack = "", needle = ""</span><br><span class="line">Output: 0</span><br></pre></td></tr></table></figure>
|
|
|
|
<h3 id="题解"><a href="#题解" class="headerlink" title="题解"></a>题解</h3><p>字符串比较其实是写代码里永恒的主题，底层的编译器等处理肯定需要字符串对比，像 kmp 算法也是很厉害</p>
|
|
<h4 id="code"><a href="#code" class="headerlink" title="code"></a>code</h4><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">int</span> <span class="title function_">strStr</span><span class="params">(String haystack, String needle)</span> {</span><br><span class="line"> <span class="comment">// 如果两个字符串都为空，返回 -1</span></span><br><span class="line"> <span class="keyword">if</span> (haystack == <span class="literal">null</span> || needle == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> -<span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 如果 haystack 长度小于 needle 长度，返回 -1</span></span><br><span class="line"> <span class="keyword">if</span> (haystack.length() < needle.length()) {</span><br><span class="line"> <span class="keyword">return</span> -<span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 如果 needle 为空字符串，返回 0</span></span><br><span class="line"> <span class="keyword">if</span> (needle.equals(<span class="string">""</span>)) {</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 如果两者相等，返回 0</span></span><br><span class="line"> <span class="keyword">if</span> (haystack.equals(needle)) {</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">int</span> <span class="variable">needleLength</span> <span class="operator">=</span> needle.length();</span><br><span class="line"> <span class="type">int</span> <span class="variable">haystackLength</span> <span class="operator">=</span> haystack.length();</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> needleLength - <span class="number">1</span>; i <= haystackLength - <span class="number">1</span>; i++) {</span><br><span class="line"> <span class="comment">// 比较 needle 最后一个字符，倒着比较稍微节省点时间</span></span><br><span class="line"> <span class="keyword">if</span> (needle.charAt(needleLength - <span class="number">1</span>) == haystack.charAt(i)) {</span><br><span class="line"> <span class="comment">// 如果needle 是 1 的话直接可以返回 i 作为位置了</span></span><br><span class="line"> <span class="keyword">if</span> (needle.length() == <span class="number">1</span>) {</span><br><span class="line"> <span class="keyword">return</span> i;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">flag</span> <span class="operator">=</span> <span class="literal">true</span>;</span><br><span class="line"> <span class="comment">// 原来比的是 needle 的最后一个位置，然后这边从倒数第二个位置开始</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">j</span> <span class="operator">=</span> needle.length() - <span class="number">2</span>;</span><br><span class="line"> <span class="keyword">for</span> (; j >= <span class="number">0</span>; j--) {</span><br><span class="line"> <span class="comment">// 这里的 i- (needleLength - j) + 1 ) 比较绕，其实是外循环的 i 表示当前 i 位置的字符跟 needle 最后一个字符</span></span><br><span class="line"> <span class="comment">// 相同，j 在上面的循环中--，对应的 haystack 也要在 i 这个位置 -- ，对应的位置就是 i - (needleLength - j) + 1</span></span><br><span class="line"> <span class="keyword">if</span> (needle.charAt(j) != haystack.charAt(i - (needleLength - j) + <span class="number">1</span>)) {</span><br><span class="line"> flag = <span class="literal">false</span>;</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 循环完了之后，如果 flag 为 true 说明 从 i 开始倒着对比都相同，但是这里需要起始位置，就需要</span></span><br><span class="line"> <span class="comment">// i - needleLength + 1</span></span><br><span class="line"> <span class="keyword">if</span> (flag) {</span><br><span class="line"> <span class="keyword">return</span> i - needleLength + <span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 这里表示未找到</span></span><br><span class="line"> <span class="keyword">return</span> -<span class="number">1</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 053 最大子序和 ( Maximum Subarray ) 题解分析</title>
|
|
<url>/2021/11/28/Leetcode-053-%E6%9C%80%E5%A4%A7%E5%AD%90%E5%BA%8F%E5%92%8C-Maximum-Subarray-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h3 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h3><p>Given an integer array <code>nums</code>, find the contiguous subarray (containing at least one number) which has the largest sum and return its sum.</p>
|
|
<p>A <strong>subarray</strong> is a <strong>contiguous</strong> part of an array.</p>
|
|
<h4 id="示例"><a href="#示例" class="headerlink" title="示例"></a>示例</h4><p><strong>Example 1:</strong></p>
|
|
<blockquote>
|
|
<p>Input: nums = [-2,1,-3,4,-1,2,1,-5,4]<br>Output: 6<br>Explanation: [4,-1,2,1] has the largest sum = 6. </p>
|
|
</blockquote>
|
|
<p><strong>Example 2:</strong></p>
|
|
<blockquote>
|
|
<p>Input: nums = [1]<br>Output: 1 </p>
|
|
</blockquote>
|
|
<p><strong>Example 3:</strong></p>
|
|
<blockquote>
|
|
<p>Input: nums = [5,4,-1,7,8]<br>Output: 23</p>
|
|
</blockquote>
|
|
<p>说起来这个题其实非常有渊源，大学数据结构的第一个题就是这个，而最佳的算法就是传说中的 online 算法，就是遍历一次就完了，最基本的做法就是记下来所有的连续子数组，然后求出最大的那个。</p>
|
|
<h3 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">int</span> <span class="title function_">maxSubArray</span><span class="params">(<span class="type">int</span>[] nums)</span> {</span><br><span class="line"> <span class="type">int</span> <span class="variable">max</span> <span class="operator">=</span> nums[<span class="number">0</span>];</span><br><span class="line"> <span class="type">int</span> <span class="variable">sum</span> <span class="operator">=</span> nums[<span class="number">0</span>];</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">1</span>; i < nums.length; i++) {</span><br><span class="line"> <span class="comment">// 这里最重要的就是这一行了，其实就是如果前面的 sum 是小于 0 的，那么就不需要前面的 sum，反正加上了还不如不加大</span></span><br><span class="line"> sum = Math.max(nums[i], sum + nums[i]);</span><br><span class="line"> <span class="comment">// max 是用来承载最大值的</span></span><br><span class="line"> max = Math.max(max, sum);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> max;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 104 二叉树的最大深度(Maximum Depth of Binary Tree) 题解分析</title>
|
|
<url>/2020/10/25/Leetcode-104-%E4%BA%8C%E5%8F%89%E6%A0%91%E7%9A%84%E6%9C%80%E5%A4%A7%E6%B7%B1%E5%BA%A6-Maximum-Depth-of-Binary-Tree-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h2 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h2><p>给定一个二叉树，找出其最大深度。</p>
|
|
<p>二叉树的深度为根节点到最远叶子节点的最长路径上的节点数。</p>
|
|
<p>说明: 叶子节点是指没有子节点的节点。</p>
|
|
<p>示例：<br>给定二叉树 [3,9,20,null,null,15,7]，</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line"> 3</span><br><span class="line"> / \</span><br><span class="line">9 20</span><br><span class="line"> / \</span><br><span class="line"> 15 7</span><br></pre></td></tr></table></figure>
|
|
<p>返回它的最大深度 3 。</p>
|
|
<h2 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// 主体是个递归的应用</span></span><br><span class="line"><span class="keyword">public</span> <span class="type">int</span> <span class="title function_">maxDepth</span><span class="params">(TreeNode root)</span> {</span><br><span class="line"> <span class="comment">// 节点的退出条件之一</span></span><br><span class="line"> <span class="keyword">if</span> (root == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">int</span> <span class="variable">left</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="type">int</span> <span class="variable">right</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="comment">// 存在左子树，就递归左子树</span></span><br><span class="line"> <span class="keyword">if</span> (root.left != <span class="literal">null</span>) {</span><br><span class="line"> left = maxDepth(root.left);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 存在右子树，就递归右子树</span></span><br><span class="line"> <span class="keyword">if</span> (root.right != <span class="literal">null</span>) {</span><br><span class="line"> right = maxDepth(root.right);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 前面返回后，左右取大者</span></span><br><span class="line"> <span class="keyword">return</span> Math.max(left + <span class="number">1</span>, right + <span class="number">1</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<h2 id="分析"><a href="#分析" class="headerlink" title="分析"></a>分析</h2><p>其实对于树这类题，一般是以递归形式比较方便，只是要注意退出条件</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
<category>Java</category>
|
|
<category>Binary Tree</category>
|
|
<category>java</category>
|
|
<category>Binary Tree</category>
|
|
<category>DFS</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
<tag>Binary Tree</tag>
|
|
<tag>DFS</tag>
|
|
<tag>二叉树</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 1115 交替打印 FooBar ( Print FooBar Alternately *Medium* ) 题解分析</title>
|
|
<url>/2022/05/01/Leetcode-1115-%E4%BA%A4%E6%9B%BF%E6%89%93%E5%8D%B0-FooBar-Print-FooBar-Alternately-Medium-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<p>无聊想去 roll 一题就看到了有并发题，就找到了这题，其实一眼看我的想法也是用信号量，但是用 condition 应该也是可以处理的，不过这类问题好像本地有点难调，因为它好像是抽取代码执行的，跟直观的逻辑比较不一样<br>Suppose you are given the following code:</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">FooBar</span> {</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">foo</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < n; i++) {</span><br><span class="line"> print(<span class="string">"foo"</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">bar</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < n; i++) {</span><br><span class="line"> print(<span class="string">"bar"</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>The same instance of <code>FooBar</code> will be passed to two different threads:</p>
|
|
<ul>
|
|
<li>thread <code>A</code> will call <code>foo()</code>, while</li>
|
|
<li>thread <code>B</code> will call <code>bar()</code>.<br>Modify the given program to output <code>"foobar"</code> n times.</li>
|
|
</ul>
|
|
<h3 id="示例"><a href="#示例" class="headerlink" title="示例"></a>示例</h3><h4 id="Example-1"><a href="#Example-1" class="headerlink" title="Example 1:"></a>Example 1:</h4><blockquote>
|
|
<p><strong>Input</strong>: n = 1<br><strong>Output</strong>: “foobar”<br><strong>Explanation</strong>: There are two threads being fired asynchronously. One of them calls foo(), while the other calls bar().<br>“foobar” is being output 1 time. </p>
|
|
</blockquote>
|
|
<h4 id="Example-2"><a href="#Example-2" class="headerlink" title="Example 2:"></a>Example 2:</h4><blockquote>
|
|
<p><strong>Input</strong>: n = 2<br><strong>Output</strong>: “foobarfoobar”<br><strong>Explanation</strong>: “foobar” is being output 2 times. </p>
|
|
</blockquote>
|
|
<h3 id="题解"><a href="#题解" class="headerlink" title="题解"></a>题解</h3><h4 id="简析"><a href="#简析" class="headerlink" title="简析"></a>简析</h4><p>其实用信号量是很直观的，就是让打印 foo 的线程先拥有信号量，打印后就等待，给 bar 信号量 + 1，然后 bar 线程运行打印消耗 bar 信号量，再给 foo 信号量 + 1</p>
|
|
<h4 id="code"><a href="#code" class="headerlink" title="code"></a>code</h4><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">FooBar</span> {</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="type">Semaphore</span> <span class="variable">foo</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Semaphore</span>(<span class="number">1</span>);</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="type">Semaphore</span> <span class="variable">bar</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Semaphore</span>(<span class="number">0</span>);</span><br><span class="line"> <span class="keyword">private</span> <span class="type">int</span> n;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">FooBar</span><span class="params">(<span class="type">int</span> n)</span> {</span><br><span class="line"> <span class="built_in">this</span>.n = n;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">foo</span><span class="params">(Runnable printFoo)</span> <span class="keyword">throws</span> InterruptedException {</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < n; i++) {</span><br><span class="line"> foo.acquire();</span><br><span class="line"> <span class="comment">// printFoo.run() outputs "foo". Do not change or remove this line.</span></span><br><span class="line"> printFoo.run();</span><br><span class="line"> bar.release();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">bar</span><span class="params">(Runnable printBar)</span> <span class="keyword">throws</span> InterruptedException {</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < n; i++) {</span><br><span class="line"> bar.acquire();</span><br><span class="line"> <span class="comment">// printBar.run() outputs "bar". Do not change or remove this line.</span></span><br><span class="line"> printBar.run();</span><br><span class="line"> foo.release();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
<tag>Print FooBar Alternately</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 121 买卖股票的最佳时机(Best Time to Buy and Sell Stock) 题解分析</title>
|
|
<url>/2021/03/14/Leetcode-121-%E4%B9%B0%E5%8D%96%E8%82%A1%E7%A5%A8%E7%9A%84%E6%9C%80%E4%BD%B3%E6%97%B6%E6%9C%BA-Best-Time-to-Buy-and-Sell-Stock-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h3 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h3><p>You are given an array <code>prices</code> where <code>prices[i]</code> is the price of a given stock on the i<sup>th</sup> day.</p>
|
|
<p>You want to maximize your profit by choosing a <strong>single day</strong> to buy one stock and choosing a <strong>different day in the future</strong> to sell that stock.</p>
|
|
<p>Return the maximum profit you can achieve from this transaction. If you cannot achieve any profit, return <code>0</code>.</p>
|
|
<p>给定一个数组 <code>prices</code> ，它的第 <code>i</code> 个元素 <code>prices[i]</code> 表示一支给定股票第 <code>i</code> 天的价格。</p>
|
|
<p>你只能选择 <strong>某一天</strong> 买入这只股票，并选择在 <strong>未来的某一个不同的日子</strong> 卖出该股票。设计一个算法来计算你所能获取的最大利润。</p>
|
|
<p>返回你可以从这笔交易中获取的最大利润。如果你不能获取任何利润，返回 <code>0</code> 。</p>
|
|
<h3 id="简单分析"><a href="#简单分析" class="headerlink" title="简单分析"></a>简单分析</h3><p>其实这个跟二叉树的最长路径和有点类似，需要找到整体的最大收益，但是在迭代过程中需要一个当前的值</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">int</span> <span class="variable">maxSofar</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"><span class="keyword">public</span> <span class="type">int</span> <span class="title function_">maxProfit</span><span class="params">(<span class="type">int</span>[] prices)</span> {</span><br><span class="line"> <span class="keyword">if</span> (prices.length <= <span class="number">1</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">int</span> <span class="variable">maxIn</span> <span class="operator">=</span> prices[<span class="number">0</span>];</span><br><span class="line"> <span class="type">int</span> <span class="variable">maxOut</span> <span class="operator">=</span> prices[<span class="number">0</span>];</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">1</span>; i < prices.length; i++) {</span><br><span class="line"> <span class="keyword">if</span> (maxIn > prices[i]) {</span><br><span class="line"> <span class="comment">// 当循环当前值小于之前的买入值时就当成买入值，同时卖出也要更新</span></span><br><span class="line"> maxIn = prices[i];</span><br><span class="line"> maxOut = prices[i];</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (prices[i] > maxOut) {</span><br><span class="line"> <span class="comment">// 表示一个可卖出点，即比买入值高时</span></span><br><span class="line"> maxOut = prices[i];</span><br><span class="line"> <span class="comment">// 需要设置一个历史值</span></span><br><span class="line"> maxSofar = Math.max(maxSofar, maxOut - maxIn);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> maxSofar;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<h3 id="总结下"><a href="#总结下" class="headerlink" title="总结下"></a>总结下</h3><p>一开始看到 easy 就觉得是很简单，就没有 maxSofar ，但是一提交就出现问题了<br>对于<code>[2, 4, 1]</code>这种就会变成 0，所以还是需要一个历史值来存放历史最大值，这题有点动态规划的意思</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
<category>Java</category>
|
|
<category>DP</category>
|
|
<category>java</category>
|
|
<category>DP</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
<tag>DP</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 105 从前序与中序遍历序列构造二叉树(Construct Binary Tree from Preorder and Inorder Traversal) 题解分析</title>
|
|
<url>/2020/12/13/Leetcode-105-%E4%BB%8E%E5%89%8D%E5%BA%8F%E4%B8%8E%E4%B8%AD%E5%BA%8F%E9%81%8D%E5%8E%86%E5%BA%8F%E5%88%97%E6%9E%84%E9%80%A0%E4%BA%8C%E5%8F%89%E6%A0%91-Construct-Binary-Tree-from-Preorder-and-Inorder-Traversal-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h2 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h2><p>Given preorder and inorder traversal of a tree, construct the binary tree.<br>给定一棵树的前序和中序遍历，构造出一棵二叉树</p>
|
|
<h3 id="注意"><a href="#注意" class="headerlink" title="注意"></a>注意</h3><p>You may assume that duplicates do not exist in the tree.<br>你可以假设树中没有重复的元素。(PS: 不然就没法做了呀)</p>
|
|
<h3 id="例子"><a href="#例子" class="headerlink" title="例子:"></a>例子:</h3><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">preorder = [3,9,20,15,7]</span><br><span class="line">inorder = [9,3,15,20,7]</span><br></pre></td></tr></table></figure>
|
|
<p>返回的二叉树</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line"> 3</span><br><span class="line"> / \</span><br><span class="line">9 20</span><br><span class="line"> / \</span><br><span class="line"> 15 7</span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
<h2 id="简要分析"><a href="#简要分析" class="headerlink" title="简要分析"></a>简要分析</h2><p>看到这个题可以想到一个比较常规的解法就是递归拆树，前序就是根左右，中序就是左根右，然后就是通过前序已经确定的根在中序中找到，然后去划分左右子树，这个例子里是 3，找到中序中的位置，那么就可以确定，9 是左子树，15,20,7是右子树，然后对应的可以根据左右子树的元素数量在前序中划分左右子树，再继续递归就行</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">Solution</span> {</span><br><span class="line"> <span class="keyword">public</span> TreeNode <span class="title function_">buildTree</span><span class="params">(<span class="type">int</span>[] preorder, <span class="type">int</span>[] inorder)</span> {</span><br><span class="line"> <span class="comment">// 获取下数组长度</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">n</span> <span class="operator">=</span> preorder.length;</span><br><span class="line"> <span class="comment">// 排除一下异常和边界</span></span><br><span class="line"> <span class="keyword">if</span> (n != inorder.length) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (n == <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (n == <span class="number">1</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">TreeNode</span>(preorder[<span class="number">0</span>]);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 获得根节点</span></span><br><span class="line"> <span class="type">TreeNode</span> <span class="variable">node</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">TreeNode</span>(preorder[<span class="number">0</span>]);</span><br><span class="line"> <span class="type">int</span> <span class="variable">pos</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="comment">// 找到中序中的位置</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < inorder.length; i++) {</span><br><span class="line"> <span class="keyword">if</span> (node.val == inorder[i]) {</span><br><span class="line"> pos = i;</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 划分左右再进行递归，注意下`Arrays.copyOfRange`的用法</span></span><br><span class="line"> node.left = buildTree(Arrays.copyOfRange(preorder, <span class="number">1</span>, pos + <span class="number">1</span>), Arrays.copyOfRange(inorder, <span class="number">0</span>, pos));</span><br><span class="line"> node.right = buildTree(Arrays.copyOfRange(preorder, pos + <span class="number">1</span>, n), Arrays.copyOfRange(inorder, pos + <span class="number">1</span>, n));</span><br><span class="line"> <span class="keyword">return</span> node;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
<category>Java</category>
|
|
<category>Binary Tree</category>
|
|
<category>java</category>
|
|
<category>Binary Tree</category>
|
|
<category>DFS</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
<tag>Binary Tree</tag>
|
|
<tag>二叉树</tag>
|
|
<tag>递归</tag>
|
|
<tag>Preorder Traversal</tag>
|
|
<tag>Inorder Traversal</tag>
|
|
<tag>前序</tag>
|
|
<tag>中序</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 124 二叉树中的最大路径和(Binary Tree Maximum Path Sum) 题解分析</title>
|
|
<url>/2021/01/24/Leetcode-124-%E4%BA%8C%E5%8F%89%E6%A0%91%E4%B8%AD%E7%9A%84%E6%9C%80%E5%A4%A7%E8%B7%AF%E5%BE%84%E5%92%8C-Binary-Tree-Maximum-Path-Sum-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h3 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h3><p>A <strong>path</strong> in a binary tree is a sequence of nodes where each pair of adjacent nodes in the sequence has an edge connecting them. A node can only appear in the sequence <strong>at most once</strong>. Note that the path does not need to pass through the root.</p>
|
|
<p>The <strong>path sum</strong> of a path is the sum of the node’s values in the path.</p>
|
|
<p>Given the <code>root</code> of a binary tree, return the maximum <strong>path sum</strong> of any path.</p>
|
|
<p><strong>路径</strong> 被定义为一条从树中任意节点出发，沿父节点-子节点连接，达到任意节点的序列。该路径 <strong>至少包含一个</strong> 节点，且不一定经过根节点。</p>
|
|
<p><strong>路径和</strong> 是路径中各节点值的总和。</p>
|
|
<p>给你一个二叉树的根节点 <code>root</code> ，返回其 <strong>最大路径和</strong></p>
|
|
<h3 id="简要分析"><a href="#简要分析" class="headerlink" title="简要分析"></a>简要分析</h3><p>其实这个题目会被误解成比较简单，左子树最大的，或者右子树最大的，或者两边加一下，仔细想想都不对，其实有可能是产生于左子树中，或者右子树中，这两个都是指跟左子树根还有右子树根没关系的，这么说感觉不太容易理解，画个图<br><img data-src="https://img.nicksxs.me/uPic/SfxIEy.png"><br>可以看到图里，其实最长路径和是左边这个子树组成的，跟根节点还有右子树完全没关系，然后再想一种情况，如果是整棵树就是图中的左子树，那么这个最长路径和就是左子树加右子树加根节点了，所以不是我一开始想得那么简单，在代码实现中也需要一些技巧</p>
|
|
<h3 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">int</span> <span class="variable">ansNew</span> <span class="operator">=</span> Integer.MIN_VALUE;</span><br><span class="line"><span class="keyword">public</span> <span class="type">int</span> <span class="title function_">maxPathSum</span><span class="params">(TreeNode root)</span> {</span><br><span class="line"> maxSumNew(root);</span><br><span class="line"> <span class="keyword">return</span> ansNew;</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"><span class="keyword">public</span> <span class="type">int</span> <span class="title function_">maxSumNew</span><span class="params">(TreeNode root)</span> {</span><br><span class="line"> <span class="keyword">if</span> (root == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 这里是个简单的递归，就是去递归左右子树，但是这里其实有个概念，当这样处理时，其实相当于把子树的内部的最大路径和已经算出来了</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">left</span> <span class="operator">=</span> maxSumNew(root.left);</span><br><span class="line"> <span class="type">int</span> <span class="variable">right</span> <span class="operator">=</span> maxSumNew(root.right);</span><br><span class="line"> <span class="comment">// 这里前面我有点没想明白，但是看到 ansNew 的比较，其实相当于，返回的是三种情况里的最大值，一个是左子树+根，一个是右子树+根，一个是单独根节点，</span></span><br><span class="line"> <span class="comment">// 这样这个递归的返回才会有意义，不然像原来的方法，它可能是跳着的，但是这种情况其实是借助于 ansNew 这个全局的最大值，因为原来我觉得要比较的是</span></span><br><span class="line"> <span class="comment">// left, right, left + root , right + root, root, left + right + root 这些的最大值，这里是分成了两个阶段，left 跟 right 的最大值已经在上面的</span></span><br><span class="line"> <span class="comment">// 调用过程中赋值给 ansNew 了 </span></span><br><span class="line"> <span class="type">int</span> <span class="variable">currentSum</span> <span class="operator">=</span> Math.max(Math.max(root.val + left , root.val + right), root.val);</span><br><span class="line"> <span class="comment">// 这边返回的是 currentSum，然后再用它跟 left + right + root 进行对比，然后再去更新 ans</span></span><br><span class="line"> <span class="comment">// PS: 有个小点也是这边的破局点，就是这个 ansNew</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">res</span> <span class="operator">=</span> Math.max(left + right + root.val, currentSum);</span><br><span class="line"> ans = Math.max(res, ans);</span><br><span class="line"> <span class="keyword">return</span> currentSum;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这里非常重要的就是 ansNew 是最后的一个结果，而对于 maxSumNew 这个函数的返回值其实是需要包含了一个连续结果，因为要返回继续去算路径和，所以返回的是 currentSum，最终结果是 ansNew</p>
|
|
<h3 id="结果图"><a href="#结果图" class="headerlink" title="结果图"></a>结果图</h3><p>难得有个 100%，贴个图哈哈<br><img data-src="https://img.nicksxs.me/uPic/iipgv0.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
<category>Java</category>
|
|
<category>Binary Tree</category>
|
|
<category>java</category>
|
|
<category>Binary Tree</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
<tag>Binary Tree</tag>
|
|
<tag>二叉树</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Java 线程池系列-第一篇</title>
|
|
<url>/2024/02/11/Java-%E7%BA%BF%E7%A8%8B%E6%B1%A0%E7%B3%BB%E5%88%97-%E7%AC%AC%E4%B8%80%E7%AF%87/</url>
|
|
<content><![CDATA[<p>这一篇我们继续聊线程池，一般线程池会介绍我们的参数，我先不一样一些<br>我们先来翻译一下</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"> An {<span class="meta">@link</span> ExecutorService} that executes each submitted task using</span><br><span class="line"> one of possibly several pooled threads, normally configured</span><br><span class="line"> using {<span class="meta">@link</span> Executors} factory methods.</span><br><span class="line"> <span class="comment">// executorService 使用一个线程池中的线程来执行每个提交的任务，经常使用 Executors 工厂方法</span></span><br><span class="line"></span><br><span class="line"> <p>Thread pools address two different problems: they usually</span><br><span class="line"> provide improved performance when executing large numbers of</span><br><span class="line"> asynchronous tasks, due to reduced per-task invocation overhead,</span><br><span class="line"> and they provide a means of bounding and managing the resources,</span><br><span class="line"> including threads, consumed when executing a collection of tasks.</span><br><span class="line"> Each {<span class="meta">@code</span> ThreadPoolExecutor} also maintains some basic</span><br><span class="line"> statistics, such as the number of completed tasks.</span><br><span class="line"> <span class="comment">// 线程池解决两个不同的问题，通常是为了在提交大量异步任务时，通过减少每个任务的调度开销来提高性能</span></span><br><span class="line"> <span class="comment">// 并且提供了一种管理和约束资源，包括线程消耗的方法，每个 ThreadPoolExecutor还会管理一些基础的统计信息</span></span><br><span class="line"> <span class="comment">// 比如完成的任务数量</span></span><br><span class="line"> </span><br><span class="line"></span><br><span class="line"> <p>To be useful across a wide range of contexts, <span class="built_in">this</span> <span class="keyword">class</span></span><br><span class="line"> <span class="title class_">provides</span> many adjustable parameters and extensibility</span><br><span class="line"> hooks. However, programmers are urged to use the more convenient</span><br><span class="line"> {<span class="meta">@link</span> Executors} factory methods {<span class="meta">@link</span></span><br><span class="line"> Executors#newCachedThreadPool} (unbounded thread pool, with</span><br><span class="line"> automatic thread reclamation), {<span class="meta">@link</span> Executors#newFixedThreadPool}</span><br><span class="line"> (fixed size thread pool) and {<span class="meta">@link</span></span><br><span class="line"> Executors#newSingleThreadExecutor} (single background thread), that</span><br><span class="line"> preconfigure settings <span class="keyword">for</span> the most common usage</span><br><span class="line"> scenarios. Otherwise, use the following guide when manually</span><br><span class="line"> configuring and tuning <span class="built_in">this</span> class:</span><br><span class="line"> <span class="comment">// 为了在各种情况下都能派上用场，该类提供了许多可调整的参数和扩展钩子。不过，我们建议程序员使用更方便的 Executors 工厂方法 Executors.</span></span><br><span class="line"> <span class="comment">// newCachedThreadPool（无限制线程池，可自动回收线程）、Executors.newFixedThreadPool（固定大小线程池）和 Executors.</span></span><br><span class="line"> <span class="comment">// newSingleThreadExecutor（单后台线程），这些方法可为最常见的使用场景预先配置设置。否则，在手动配置和调整该类时，请使用以下指南：</span></span><br><span class="line"></span><br><span class="line"> Core and maximum pool sizes</span><br><span class="line"> <span class="comment">// 核心和最大线程池大小</span></span><br><span class="line"> A ThreadPoolExecutor will automatically adjust the pool <span class="title function_">size</span> <span class="params">(see getPoolSize)</span> according to the bounds set by <span class="title function_">corePoolSize</span> <span class="params">(see getCorePoolSize)</span> and <span class="title function_">maximumPoolSize</span> <span class="params">(see getMaximumPoolSize)</span>. When a <span class="keyword">new</span> <span class="title class_">task</span> is submitted in method <span class="title function_">execute</span><span class="params">(Runnable)</span>, and fewer than corePoolSize threads are running, a <span class="keyword">new</span> <span class="title class_">thread</span> is created to handle the request, even <span class="keyword">if</span> other worker threads are idle. If there are more than corePoolSize but less than maximumPoolSize threads running, a <span class="keyword">new</span> <span class="title class_">thread</span> will be created only <span class="keyword">if</span> the queue is full. By setting corePoolSize and maximumPoolSize the same, you create a fixed-size thread pool. By setting maximumPoolSize to an essentially unbounded value such as Integer.MAX_VALUE, you allow the pool to accommodate an arbitrary number of concurrent tasks. Most typically, core and maximum pool sizes are set only upon construction, but they may also be changed dynamically using setCorePoolSize and setMaximumPoolSize.</span><br><span class="line"> <span class="comment">// ThreadPoolExecutor 会根据 corePoolSize（参见 getCorePoolSize）和 maximumPoolSize（参见 getMaximumPoolSize）设置的界限自动调整池大小（参见 getPoolSize）。在方法 execute(Runnable) 中提交新任务时，如果运行的线程少于 corePoolSize，即使其他工作线程处于空闲状态，也会创建一个新线程来处理请求。如果运行的线程多于 corePoolSize 但少于 maximumPoolSize，则只有在队列已满时才会创建新线程。如果将 corePoolSize 和 maximumPoolSize 设置为相同，就可以创建一个固定大小的线程池。如果将 maximumPoolSize 设置为 Integer.MAX_VALUE 等基本无限制的值，就可以让池容纳任意数量的并发任务。通常，核心和最大线程池大小只在构建时设置，但也可以使用 setCorePoolSize 和 setMaximumPoolSize 动态更改。</span></span><br><span class="line"></span><br><span class="line">On-demand construction</span><br><span class="line"><span class="comment">// 按需构建</span></span><br><span class="line"> By <span class="keyword">default</span>, even core threads are initially created and started only when <span class="keyword">new</span> <span class="title class_">tasks</span> arrive, but <span class="built_in">this</span> can be overridden dynamically using method prestartCoreThread or prestartAllCoreThreads. You probably want to prestart threads <span class="keyword">if</span> you construct the pool with a non-empty queue.</span><br><span class="line"> <span class="comment">// 默认情况下，即使是核心线程也是只有当新任务到来时才会初始创建和启动，但这可以使用方法 prestartCoreThread 或 prestartAllCoreThreads 进行动态重载。如果使用非空队列构建线程池，则可能需要预启动线程。</span></span><br><span class="line">Creating <span class="keyword">new</span> <span class="title class_">threads</span></span><br><span class="line"><span class="comment">// 创建新线程</span></span><br><span class="line"> <span class="comment">/** New threads are created using a ThreadFactory. If not otherwise specified, a Executors.defaultThreadFactory is used, that creates threads to all be in the same ThreadGroup and with the same NORM_PRIORITY priority and non-daemon status. By supplying a different ThreadFactory, you can alter the thread's name, thread group, priority, daemon status, etc. If a ThreadFactory fails to create a thread when asked by returning null from newThread, the executor will continue, but might not be able to execute any tasks. Threads should possess the "modifyThread" RuntimePermission. If worker threads or other threads using the pool do not possess this permission, service may be degraded: configuration changes may not take effect in a timely manner, and a shutdown pool may remain in a state in which termination is possible but not completed. */</span></span><br><span class="line"> <span class="comment">// 新线程是使用线程工厂（ThreadFactory）创建的。如果未另行指定，将使用 Executors.defaultThreadFactory 创建线程，创建的线程将全部位于同一线程组，并具有相同的 NORM_PRIORITY 优先级和非守护进程状态。通过提供不同的 ThreadFactory，可以更改线程的名称、线程组、优先级、守护进程状态等。如果线程工厂无法按要求创建线程，从 newThread 返回空值，执行器将继续运行，但可能无法执行任何任务。线程应拥有 "modifyThread "运行时权限。如果工作线程或使用池的其他线程不具备此权限，服务可能会降级：配置更改可能无法及时生效，关闭的池可能会一直处于可以终止但无法完成的状态。</span></span><br><span class="line">Keep-alive times</span><br><span class="line"><span class="comment">// 保活时间</span></span><br><span class="line"> If the pool currently has more than corePoolSize threads, excess threads will be terminated <span class="keyword">if</span> they have been idle <span class="keyword">for</span> more than the <span class="title function_">keepAliveTime</span> <span class="params">(see getKeepAliveTime(TimeUnit)</span>). This provides a means of reducing resource consumption when the pool is not being actively used. If the pool becomes more active later, <span class="keyword">new</span> <span class="title class_">threads</span> will be constructed. This parameter can also be changed dynamically using method <span class="title function_">setKeepAliveTime</span><span class="params">(<span class="type">long</span>, TimeUnit)</span>. Using a value of Long.MAX_VALUE TimeUnit.NANOSECONDS effectively disables idle threads from ever terminating prior to shut down. By <span class="keyword">default</span>, the keep-alive policy applies only when there are more than corePoolSize threads. But method <span class="title function_">allowCoreThreadTimeOut</span><span class="params">(<span class="type">boolean</span>)</span> can be used to apply <span class="built_in">this</span> time-out policy to core threads as well, so <span class="type">long</span> as the keepAliveTime value is non-zero.</span><br><span class="line"> <span class="comment">// 如果当前池中的线程数超过 corePoolSize，多余的线程在闲置时间超过 keepAliveTime（请参阅 getKeepAliveTime(TimeUnit)）时将被终止。这提供了一种在线程池未被频繁使用时减少资源消耗的方法。如果以后池变得更加活跃，就会构建新线程。也可以使用方法 setKeepAliveTime(long, TimeUnit) 动态更改该参数。使用 Long.MAX_VALUE TimeUnit.NANOSECONDS 值可有效禁止闲置线程在关闭前终止。默认情况下，只有当线程数超过 corePoolSize 时，才会采用保持连接策略。但只要 keepAliveTime 值不为零，就可以使用方法 allowCoreThreadTimeOut(boolean) 将超时策略也应用到核心线程。</span></span><br><span class="line">Queuing</span><br><span class="line"><span class="comment">// 队列</span></span><br><span class="line"> Any BlockingQueue may be used to transfer and hold submitted tasks. The use of <span class="built_in">this</span> queue interacts with pool sizing:</span><br><span class="line"> <span class="comment">// 任何阻塞队列（BlockingQueue）都可用于传输和保留已提交的任务。该队列的使用与线程池大小有关：</span></span><br><span class="line"> If fewer than corePoolSize threads are running, the Executor always prefers adding a <span class="keyword">new</span> <span class="title class_">thread</span> rather than queuing.</span><br><span class="line"> <span class="comment">// 如果运行的线程少于 corePoolSize，执行器总是倾向于添加新线程而不是队列。</span></span><br><span class="line"> If corePoolSize or more threads are running, the Executor always prefers queuing a request rather than adding a <span class="keyword">new</span> <span class="title class_">thread</span>.</span><br><span class="line"> <span class="comment">// 如果运行的线程数大于或等于 corePoolSize，执行器总是倾向于将请求排队，而不是添加新线程。</span></span><br><span class="line"> If a request cannot be queued, a <span class="keyword">new</span> <span class="title class_">thread</span> is created unless <span class="built_in">this</span> would exceed maximumPoolSize, in which <span class="keyword">case</span>, the task will be rejected.</span><br><span class="line"> <span class="comment">// 如果请求无法进队列，执行器会创建一个新线程，除非这样做会超过最大线程池大小（maximumPoolSize），在这种情况下，任务会被拒绝。</span></span><br><span class="line"> There are three general strategies <span class="keyword">for</span> queuing:</span><br><span class="line"> <span class="comment">// 入队一般有三种策略：</span></span><br><span class="line"> Direct handoffs. A good <span class="keyword">default</span> choice <span class="keyword">for</span> a work queue is a SynchronousQueue that hands off tasks to threads without otherwise holding them. Here, an attempt to queue a task will fail <span class="keyword">if</span> no threads are immediately available to run it, so a <span class="keyword">new</span> <span class="title class_">thread</span> will be constructed. This policy avoids lockups when handling sets of requests that might have internal dependencies. Direct handoffs generally require unbounded maximumPoolSizes to avoid rejection of <span class="keyword">new</span> <span class="title class_">submitted</span> tasks. This in turn admits the possibility of unbounded thread growth when commands <span class="keyword">continue</span> to arrive on average faster than they can be processed.</span><br><span class="line"> <span class="comment">// 直接交接。工作队列的一个很好的默认选择是同步队列（SynchronousQueue），它可以将任务移交给线程，而不会以其他方式保留任务。在这里，如果没有线程可立即运行任务，则队列任务的尝试将失败，因此会构建一个新的线程。在处理可能存在内部依赖关系的请求集时，这种策略可以避免死锁。直接交接通常需要无限制的最大线程池大小，以避免新提交的任务被拒绝。反过来，当命令的平均到达速度超过其处理速度时，就有可能导致线程无限制增长。</span></span><br><span class="line"> <span class="comment">/** Unbounded queues. Using an unbounded queue (for example a LinkedBlockingQueue without a predefined capacity) will cause new tasks to wait in the queue when all corePoolSize threads are busy. Thus, no more than corePoolSize threads will ever be created. (And the value of the maximumPoolSize therefore doesn't have any effect.) This may be appropriate when each task is completely independent of others, so tasks cannot affect each others execution; for example, in a web page server. While this style of queuing can be useful in smoothing out transient bursts of requests, it admits the possibility of unbounded work queue growth when commands continue to arrive on average faster than they can be processed. */</span></span><br><span class="line"> <span class="comment">// 无界队列。使用无界队列（例如没有预定义容量的 LinkedBlockingQueue）会导致新任务在所有 corePoolSize 线程都繁忙时在队列中等待。因此，创建的线程不会超过 corePoolSize。(当每个任务都完全独立于其他任务，因此任务之间不会相互影响执行时，例如在网页服务器中，这种方式可能比较合适。虽然这种队列方式在平滑瞬时突发请求方面很有用，但当命令的平均到达速度持续超过其处理速度时，就会导致工作队列无限制地增长。</span></span><br><span class="line"> Bounded queues. A bounded <span class="title function_">queue</span> <span class="params">(<span class="keyword">for</span> example, an ArrayBlockingQueue)</span> helps prevent resource exhaustion when used with finite maximumPoolSizes, but can be more difficult to tune and control. Queue sizes and maximum pool sizes may be traded off <span class="keyword">for</span> each other: Using large queues and small pools minimizes CPU usage, OS resources, and context-switching overhead, but can lead to artificially low throughput. If tasks frequently <span class="title function_">block</span> <span class="params">(<span class="keyword">for</span> example <span class="keyword">if</span> they are I/O bound)</span>, a system may be able to schedule time <span class="keyword">for</span> more threads than you otherwise allow. Use of small queues generally <span class="keyword">requires</span> larger pool sizes, which keeps CPUs busier but may encounter unacceptable scheduling overhead, which also decreases throughput.</span><br><span class="line"> <span class="comment">// 有界队列 有界队列（例如 ArrayBlockingQueue）与有限的最大线程池大小一起使用时，有助于防止资源耗尽，但可能更难调整和控制。队列大小和最大池大小可以相互权衡： 使用大队列和小池可以最大限度地减少 CPU 占用率、操作系统资源和上下文切换开销，但会导致人为的低吞吐量。如果任务经常出现阻塞（例如，如果任务受 I/O 约束），系统可能会为更多线程安排时间，而不考虑其他因素。使用小队列通常需要更大的池规模，这将使 CPU 更繁忙，但可能会遇到无法接受的调度开销，这也会降低吞吐量。</span></span><br><span class="line">Rejected tasks</span><br><span class="line"><span class="comment">// 被拒绝的任务</span></span><br><span class="line"> New tasks submitted in method <span class="title function_">execute</span><span class="params">(Runnable)</span> will be rejected when the Executor has been shut down, and also when the Executor uses finite bounds <span class="keyword">for</span> both maximum threads and work queue capacity, and is saturated. In either <span class="keyword">case</span>, the execute method invokes the RejectedExecutionHandler.rejectedExecution(Runnable, ThreadPoolExecutor) method of its RejectedExecutionHandler. Four predefined handler policies are provided:</span><br><span class="line"> <span class="comment">// 如果执行器已关闭，或者执行器使用了最大线程数和工作队列容量的有限界限，并已达到饱和，那么在 execute(Runnable) 方法中提交的新任务将被拒绝。在这两种情况下，execute 方法都会调用其 RejectedExecutionHandler 的 RejectedExecutionHandler.rejectedExecution(Runnable, ThreadPoolExecutor) 方法。提供了四种预定义的处理程序策略：</span></span><br><span class="line"> In the <span class="keyword">default</span> ThreadPoolExecutor.AbortPolicy, the handler <span class="keyword">throws</span> a runtime RejectedExecutionException upon rejection.</span><br><span class="line"> <span class="comment">// 在默认的 ThreadPoolExecutor.AbortPolicy 中，处理程序会在拒绝时抛出运行时 RejectedExecutionException。</span></span><br><span class="line"> In ThreadPoolExecutor.CallerRunsPolicy, the thread that invokes execute itself runs the task. This provides a simple feedback control mechanism that will slow down the rate that <span class="keyword">new</span> <span class="title class_">tasks</span> are submitted.</span><br><span class="line"> <span class="comment">// 在 ThreadPoolExecutor.CallerRunsPolicy 中，调用执行本身的线程会运行任务。这提供了一个简单的反馈控制机制，可减慢提交新任务的速度。</span></span><br><span class="line"> In ThreadPoolExecutor.DiscardPolicy, a task that cannot be executed is simply dropped.</span><br><span class="line"> <span class="comment">// 在 ThreadPoolExecutor.DiscardPolicy 中，无法执行的任务会被直接丢弃。</span></span><br><span class="line"> In ThreadPoolExecutor.DiscardOldestPolicy, <span class="keyword">if</span> the executor is not shut down, the task at the head of the work queue is dropped, and then execution is <span class="title function_">retried</span> <span class="params">(which can fail again, causing <span class="built_in">this</span> to be repeated.)</span></span><br><span class="line"> <span class="comment">// 在 ThreadPoolExecutor.DiscardOldestPolicy 中，如果执行器没有关闭，工作队列头部的任务就会被丢弃，然后重新执行（可能会再次失败，导致重复执行）。</span></span><br><span class="line"> It is possible to define and use other kinds of RejectedExecutionHandler classes. Doing so <span class="keyword">requires</span> some care especially when policies are designed to work only under particular capacity or queuing policies.</span><br><span class="line"> <span class="comment">// 我们还可以定义和使用其他类型的 RejectedExecutionHandler 类。这样做需要小心谨慎，尤其是当策略设计为仅在特定容量或队列策略下运行时。</span></span><br><span class="line">Hook methods</span><br><span class="line"><span class="comment">// 钩子方法</span></span><br><span class="line"> This <span class="keyword">class</span> <span class="title class_">provides</span> <span class="keyword">protected</span> overridable <span class="title function_">beforeExecute</span><span class="params">(Thread, Runnable)</span> and <span class="title function_">afterExecute</span><span class="params">(Runnable, Throwable)</span> methods that are called before and after execution of each task. These can be used to manipulate the execution environment; <span class="keyword">for</span> example, reinitializing ThreadLocals, gathering statistics, or adding log entries. Additionally, method terminated can be overridden to perform any special processing that needs to be done once the Executor has fully terminated.</span><br><span class="line"> <span class="comment">// 该类提供 protected 的可重载 beforeExecute(Thread, Runnable) 和 afterExecute(Runnable, Throwable) 方法，可在每个任务执行前后调用。这些方法可用于操纵执行环境，例如，重新初始化线程位置、收集统计数据或添加日志条目。此外，还可以重写终止方法，以便在执行器完全终止后执行任何需要进行的特殊处理。(一些动态线程池技术就用到了这两个方法)</span></span><br><span class="line"> If hook or callback methods <span class="keyword">throw</span> exceptions, internal worker threads may in turn fail and abruptly terminate.</span><br><span class="line"> <span class="comment">// 如果钩子或回调方法抛出异常，内部工作线程可能会失败并突然终止。</span></span><br><span class="line">Queue maintenance</span><br><span class="line"><span class="comment">// 队列维护</span></span><br><span class="line"> Method <span class="title function_">getQueue</span><span class="params">()</span> allows access to the work queue <span class="keyword">for</span> purposes of monitoring and debugging. Use of <span class="built_in">this</span> method <span class="keyword">for</span> any other purpose is strongly discouraged. Two supplied methods, remove(Runnable) and purge are available to assist in storage reclamation when large numbers of queued tasks become cancelled.</span><br><span class="line"> <span class="comment">// 方法 getQueue() 允许访问工作队列，以便进行监控和调试。强烈建议不要将此方法用于任何其他目的。当大量队列任务被取消时，可使用 remove(Runnable) 和 purge 这两个提供的方法来帮助回收存储空间。(不建议直接将获取到的队列进行修改操作)</span></span><br><span class="line">Finalization</span><br><span class="line"><span class="comment">// 终止</span></span><br><span class="line"> A pool that is no longer referenced in a program AND has no remaining threads will be shutdown automatically. If you would like to ensure that unreferenced pools are reclaimed even <span class="keyword">if</span> users forget to call shutdown, then you must arrange that unused threads eventually die, by setting appropriate keep-alive times, using a lower bound of zero core threads and/or setting <span class="title function_">allowCoreThreadTimeOut</span><span class="params">(<span class="type">boolean</span>)</span>.</span><br><span class="line"> <span class="comment">// 程序中不再引用且没有剩余线程的池将自动关闭。如果要确保即使用户忘记调用 shutdown 也能回收未引用的池，则必须通过设置适当的保持连接时间、使用零核心线程下限和/或设置 allowCoreThreadTimeOut(boolean) 来使得未使用的线程最终被停止。</span></span><br></pre></td></tr></table></figure>
|
|
<p>通过前面的简单翻译我们就能大致略窥得线程池配置全貌，继续看一个构造方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">ThreadPoolExecutor</span><span class="params">(<span class="type">int</span> corePoolSize,</span></span><br><span class="line"><span class="params"> <span class="type">int</span> maximumPoolSize,</span></span><br><span class="line"><span class="params"> <span class="type">long</span> keepAliveTime,</span></span><br><span class="line"><span class="params"> TimeUnit unit,</span></span><br><span class="line"><span class="params"> BlockingQueue<Runnable> workQueue)</span> {</span><br><span class="line"> <span class="built_in">this</span>(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue,</span><br><span class="line"> Executors.defaultThreadFactory(), defaultHandler);</span><br><span class="line">}</span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">RejectedExecutionHandler</span> <span class="variable">defaultHandler</span> <span class="operator">=</span></span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">AbortPolicy</span>();</span><br></pre></td></tr></table></figure>
|
|
<p>这个构造方法就主要配置了 <code>corePoolSize</code> 核心线程数，<code>maximumPoolSize</code> 最大线程数, <code>keepAliveTime</code> 线程保活时间，<code>unit</code> 时间单位，<code>workQueue</code> 工作队列，而传入后可以看到使用了默认的线程工厂和默认的拒绝处理器</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">ThreadPoolExecutor</span><span class="params">(<span class="type">int</span> corePoolSize,</span></span><br><span class="line"><span class="params"> <span class="type">int</span> maximumPoolSize,</span></span><br><span class="line"><span class="params"> <span class="type">long</span> keepAliveTime,</span></span><br><span class="line"><span class="params"> TimeUnit unit,</span></span><br><span class="line"><span class="params"> BlockingQueue<Runnable> workQueue,</span></span><br><span class="line"><span class="params"> ThreadFactory threadFactory,</span></span><br><span class="line"><span class="params"> RejectedExecutionHandler handler)</span> {</span><br><span class="line"> <span class="keyword">if</span> (corePoolSize < <span class="number">0</span> ||</span><br><span class="line"> maximumPoolSize <= <span class="number">0</span> ||</span><br><span class="line"> maximumPoolSize < corePoolSize ||</span><br><span class="line"> keepAliveTime < <span class="number">0</span>)</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalArgumentException</span>();</span><br><span class="line"> <span class="keyword">if</span> (workQueue == <span class="literal">null</span> || threadFactory == <span class="literal">null</span> || handler == <span class="literal">null</span>)</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">NullPointerException</span>();</span><br><span class="line"> <span class="built_in">this</span>.acc = System.getSecurityManager() == <span class="literal">null</span> ?</span><br><span class="line"> <span class="literal">null</span> :</span><br><span class="line"> AccessController.getContext();</span><br><span class="line"> <span class="built_in">this</span>.corePoolSize = corePoolSize;</span><br><span class="line"> <span class="built_in">this</span>.maximumPoolSize = maximumPoolSize;</span><br><span class="line"> <span class="built_in">this</span>.workQueue = workQueue;</span><br><span class="line"> <span class="built_in">this</span>.keepAliveTime = unit.toNanos(keepAliveTime);</span><br><span class="line"> <span class="built_in">this</span>.threadFactory = threadFactory;</span><br><span class="line"> <span class="built_in">this</span>.handler = handler;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>最终会执行到这里做一些校验判断，包括几个参数校验和配置赋值</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 155 最小栈(Min Stack) 题解分析</title>
|
|
<url>/2020/12/06/Leetcode-155-%E6%9C%80%E5%B0%8F%E6%A0%88-Min-Stack-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h2 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h2><p>Design a stack that supports push, pop, top, and retrieving the minimum element in constant time.<br>设计一个栈，支持压栈，出站，获取栈顶元素，通过常数级复杂度获取栈中的最小元素</p>
|
|
<ul>
|
|
<li>push(x) – Push element x onto stack.</li>
|
|
<li>pop() – Removes the element on top of the stack.</li>
|
|
<li>top() – Get the top element.</li>
|
|
<li>getMin() – Retrieve the minimum element in the stack.</li>
|
|
</ul>
|
|
<h2 id="示例"><a href="#示例" class="headerlink" title="示例"></a>示例</h2><p>Example 1:</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Input</span><br><span class="line">["MinStack","push","push","push","getMin","pop","top","getMin"]</span><br><span class="line">[[],[-2],[0],[-3],[],[],[],[]]</span><br><span class="line"></span><br><span class="line">Output</span><br><span class="line">[null,null,null,null,-3,null,0,-2]</span><br><span class="line"></span><br><span class="line">Explanation</span><br><span class="line">MinStack minStack = new MinStack();</span><br><span class="line">minStack.push(-2);</span><br><span class="line">minStack.push(0);</span><br><span class="line">minStack.push(-3);</span><br><span class="line">minStack.getMin(); // return -3</span><br><span class="line">minStack.pop();</span><br><span class="line">minStack.top(); // return 0</span><br><span class="line">minStack.getMin(); // return -2</span><br></pre></td></tr></table></figure>
|
|
|
|
<h2 id="简要分析"><a href="#简要分析" class="headerlink" title="简要分析"></a>简要分析</h2><p>其实现在大部分语言都自带类栈的数据结构，Java 也自带 stack 这个数据结构，所以这个题的主要难点的就是常数级的获取最小元素，最开始的想法是就一个栈外加一个记录最小值的变量就行了，但是仔细一想是不行的，因为随着元素被 pop 出去，这个最小值也可能需要梗着变化，就不太好判断了，所以后面是用了一个辅助栈。</p>
|
|
<h2 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">MinStack</span> {</span><br><span class="line"> <span class="comment">// 这个作为主栈</span></span><br><span class="line"> Stack<Integer> s1 = <span class="keyword">new</span> <span class="title class_">Stack</span><>();</span><br><span class="line"> <span class="comment">// 这个作为辅助栈，放最小值的栈</span></span><br><span class="line"> Stack<Integer> s2 = <span class="keyword">new</span> <span class="title class_">Stack</span><>();</span><br><span class="line"> <span class="comment">/** initialize your data structure here. */</span></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">MinStack</span><span class="params">()</span> {</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">push</span><span class="params">(<span class="type">int</span> x)</span> {</span><br><span class="line"> <span class="comment">// 放入主栈</span></span><br><span class="line"> s1.push(x);</span><br><span class="line"> <span class="comment">// 当 s2 是空或者当前值是小于"等于" s2 栈顶时，压入辅助最小值的栈</span></span><br><span class="line"> <span class="comment">// 注意这里的"等于"非常必要，因为当最小值有多个的情况下，也需要压入栈，否则在 pop 的时候就会不对等</span></span><br><span class="line"> <span class="keyword">if</span> (s2.isEmpty() || x <= s2.peek()) {</span><br><span class="line"> s2.push(x);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">pop</span><span class="params">()</span> {</span><br><span class="line"> <span class="comment">// 首先就是主栈要 pop，然后就是第二个了，跟上面的"等于"很有关系，</span></span><br><span class="line"> <span class="comment">// 因为如果有两个最小值，如果前面等于的情况没有压栈，那这边相等的时候 pop 就会少一个了，可能就导致最小值不对了</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">x</span> <span class="operator">=</span> s1.pop();</span><br><span class="line"> <span class="keyword">if</span> (x == s2.peek()) {</span><br><span class="line"> s2.pop();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="type">int</span> <span class="title function_">top</span><span class="params">()</span> {</span><br><span class="line"> <span class="comment">// 栈顶的元素</span></span><br><span class="line"> <span class="keyword">return</span> s1.peek();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="type">int</span> <span class="title function_">getMin</span><span class="params">()</span> {</span><br><span class="line"> <span class="comment">// 辅助最小栈的栈顶</span></span><br><span class="line"> <span class="keyword">return</span> s2.peek();</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
<category>Java</category>
|
|
<category>stack</category>
|
|
<category>java</category>
|
|
<category>stack</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
<tag>stack</tag>
|
|
<tag>min stack</tag>
|
|
<tag>最小栈</tag>
|
|
<tag>leetcode 155</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 1260 二维网格迁移 ( Shift 2D Grid *Easy* ) 题解分析</title>
|
|
<url>/2022/07/22/Leetcode-1260-%E4%BA%8C%E7%BB%B4%E7%BD%91%E6%A0%BC%E8%BF%81%E7%A7%BB-Shift-2D-Grid-Easy-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h3 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h3><p>Given a 2D <code>grid</code> of size <code>m x n</code> and an integer <code>k</code>. You need to shift the <code>grid k</code> times.</p>
|
|
<p>In one shift operation:</p>
|
|
<p>Element at <code>grid[i][j]</code> moves to <code>grid[i][j + 1]</code>.<br>Element at <code>grid[i][n - 1]</code> moves to <code>grid[i + 1][0]</code>.<br>Element at <code>grid[m - 1][n - 1]</code> moves to <code>grid[0][0]</code>.<br>Return the <em>2D grid</em> after applying shift operation <code>k</code> times.</p>
|
|
<h3 id="示例"><a href="#示例" class="headerlink" title="示例"></a>示例</h3><h4 id="Example-1"><a href="#Example-1" class="headerlink" title="Example 1:"></a>Example 1:</h4><p><img data-src="https://img.nicksxs.me/uPic/A6DykH.jpg"> </p>
|
|
<blockquote>
|
|
<p><strong>Input</strong>: grid = [[1,2,3],[4,5,6],[7,8,9]], k = 1<br><strong>Output</strong>: [[9,1,2],[3,4,5],[6,7,8]] </p>
|
|
</blockquote>
|
|
<h4 id="Example-2"><a href="#Example-2" class="headerlink" title="Example 2:"></a>Example 2:</h4><p><img data-src="https://img.nicksxs.me/uPic/fcjxe9.jpg"> </p>
|
|
<blockquote>
|
|
<p><strong>Input</strong>: grid = [[3,8,1,9],[19,7,2,5],[4,6,11,10],[12,0,21,13]], k = 4<br><strong>Output</strong>: [[12,0,21,13],[3,8,1,9],[19,7,2,5],[4,6,11,10]] </p>
|
|
</blockquote>
|
|
<h4 id="Example-3"><a href="#Example-3" class="headerlink" title="Example 3:"></a>Example 3:</h4><blockquote>
|
|
<p><strong>Input</strong>: grid = [[1,2,3],[4,5,6],[7,8,9]], k = 9<br><strong>Output</strong>: [[1,2,3],[4,5,6],[7,8,9]] </p>
|
|
</blockquote>
|
|
<h3 id="提示"><a href="#提示" class="headerlink" title="提示"></a>提示</h3><ul>
|
|
<li><code>m == grid.length</code></li>
|
|
<li><code>n == grid[i].length</code></li>
|
|
<li><code>1 <= m <= 50</code></li>
|
|
<li><code>1 <= n <= 50</code></li>
|
|
<li><code>-1000 <= grid[i][j] <= 1000</code></li>
|
|
<li><code>0 <= k <= 100</code></li>
|
|
</ul>
|
|
<h3 id="解析"><a href="#解析" class="headerlink" title="解析"></a>解析</h3><p>这个题主要是矩阵或者说数组的操作，并且题目要返回的是个 List，所以也不用原地操作，只需要找对位置就可以了，k 是多少就相当于让这个二维数组头尾衔接移动 k 个元素</p>
|
|
<h3 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> List<List<Integer>> <span class="title function_">shiftGrid</span><span class="params">(<span class="type">int</span>[][] grid, <span class="type">int</span> k)</span> {</span><br><span class="line"> <span class="comment">// 行数</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">m</span> <span class="operator">=</span> grid.length;</span><br><span class="line"> <span class="comment">// 列数</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">n</span> <span class="operator">=</span> grid[<span class="number">0</span>].length;</span><br><span class="line"> <span class="comment">// 偏移值，取下模</span></span><br><span class="line"> k = k % (m * n);</span><br><span class="line"> <span class="comment">// 反向取下数量，因为我打算直接从头填充新的矩阵</span></span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> * 比如</span></span><br><span class="line"><span class="comment"> * 1 2 3</span></span><br><span class="line"><span class="comment"> * 4 5 6</span></span><br><span class="line"><span class="comment"> * 7 8 9</span></span><br><span class="line"><span class="comment"> * 需要变成</span></span><br><span class="line"><span class="comment"> * 9 1 2</span></span><br><span class="line"><span class="comment"> * 3 4 5</span></span><br><span class="line"><span class="comment"> * 6 7 8</span></span><br><span class="line"><span class="comment"> * 就要从 9 开始填充</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">reverseK</span> <span class="operator">=</span> m * n - k;</span><br><span class="line"> List<List<Integer>> matrix = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"> <span class="comment">// 这类就是两层循环</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < m; i++) {</span><br><span class="line"> List<Integer> line = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">j</span> <span class="operator">=</span> <span class="number">0</span>; j < n; j++) {</span><br><span class="line"> <span class="comment">// 数量会随着循环迭代增长, 确认是第几个</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">currentNum</span> <span class="operator">=</span> reverseK + i * n + (j + <span class="number">1</span>);</span><br><span class="line"> <span class="comment">// 这里处理下到达矩阵末尾后减掉 m * n</span></span><br><span class="line"> <span class="keyword">if</span> (currentNum > m * n) {</span><br><span class="line"> currentNum -= m * n;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 根据矩阵列数 n 算出在原来矩阵的位置</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">last</span> <span class="operator">=</span> (currentNum - <span class="number">1</span>) % n;</span><br><span class="line"> <span class="type">int</span> <span class="variable">passLine</span> <span class="operator">=</span> (currentNum - <span class="number">1</span>) / n;</span><br><span class="line"></span><br><span class="line"> line.add(grid[passLine][last]);</span><br><span class="line"> }</span><br><span class="line"> matrix.add(line);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> matrix;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<h3 id="结果数据"><a href="#结果数据" class="headerlink" title="结果数据"></a>结果数据</h3><p><img data-src="https://img.nicksxs.me/uPic/3JcV8r.png"><br>比较慢</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
<tag>Shift 2D Grid</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 160 相交链表(intersection-of-two-linked-lists) 题解分析</title>
|
|
<url>/2021/01/10/Leetcode-160-%E7%9B%B8%E4%BA%A4%E9%93%BE%E8%A1%A8-intersection-of-two-linked-lists-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h2 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h2><p>写一个程序找出两个单向链表的交叉起始点，可能是我英语不好，图里画的其实还有一点是交叉以后所有节点都是相同的<br>Write a program to find the node at which the intersection of two singly linked lists begins.</p>
|
|
<p>For example, the following two linked lists:<br><img data-src="https://img.nicksxs.me/uPic/nkha2z.png"><br>begin to intersect at node c1.</p>
|
|
<h3 id="Example-1"><a href="#Example-1" class="headerlink" title="Example 1:"></a>Example 1:</h3><p><img data-src="https://img.nicksxs.me/uPic/1TrhYe.png"></p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Input: intersectVal = 8, listA = [4,1,8,4,5], listB = [5,6,1,8,4,5], skipA = 2, skipB = 3</span><br><span class="line">Output: Reference of the node with value = 8</span><br><span class="line">Input Explanation: The intersected node's value is 8 (note that this must not be 0 if the two lists intersect). From the head of A, it reads as [4,1,8,4,5]. From the head of B, it reads as [5,6,1,8,4,5]. There are 2 nodes before the intersected node in A; There are 3 nodes before the intersected node in B.</span><br></pre></td></tr></table></figure>
|
|
<h2 id="分析题解"><a href="#分析题解" class="headerlink" title="分析题解"></a>分析题解</h2><p>一开始没什么头绪，感觉只能最原始的遍历，后来看了一些文章，发现比较简单的方式就是先找两个链表的长度差，因为从相交点开始肯定是长度一致的，这是个很好的解题突破口，找到长度差以后就是先跳过长链表的较长部分，然后开始同步遍历比较 A，B 链表；</p>
|
|
<h3 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> ListNode <span class="title function_">getIntersectionNode</span><span class="params">(ListNode headA, ListNode headB)</span> {</span><br><span class="line"> <span class="keyword">if</span> (headA == <span class="literal">null</span> || headB == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 算 A 的长度</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">countA</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="type">ListNode</span> <span class="variable">tailA</span> <span class="operator">=</span> headA;</span><br><span class="line"> <span class="keyword">while</span> (tailA != <span class="literal">null</span>) {</span><br><span class="line"> tailA = tailA.next;</span><br><span class="line"> countA++;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 算 B 的长度</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">countB</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="type">ListNode</span> <span class="variable">tailB</span> <span class="operator">=</span> headB;</span><br><span class="line"> <span class="keyword">while</span> (tailB != <span class="literal">null</span>) {</span><br><span class="line"> tailB = tailB.next;</span><br><span class="line"> countB++;</span><br><span class="line"> }</span><br><span class="line"> tailA = headA;</span><br><span class="line"> tailB = headB;</span><br><span class="line"> <span class="comment">// 依据长度差，先让长的链表 tail 指针往后移</span></span><br><span class="line"> <span class="keyword">if</span> (countA > countB) {</span><br><span class="line"> <span class="keyword">while</span> (countA > countB) {</span><br><span class="line"> tailA = tailA.next;</span><br><span class="line"> countA--;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (countA < countB) {</span><br><span class="line"> <span class="keyword">while</span> (countA < countB) {</span><br><span class="line"> tailB = tailB.next;</span><br><span class="line"> countB--;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 然后以相同速度遍历两个链表比较</span></span><br><span class="line"> <span class="keyword">while</span> (tailA != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">if</span> (tailA == tailB) {</span><br><span class="line"> <span class="keyword">return</span> tailA;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> tailA = tailA.next;</span><br><span class="line"> tailB = tailB.next;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<h3 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h3><p>可能缺少这种思维，做的还是比较少，所以没法一下子反应过来，需要锻炼，我的第一反应是两重遍历，不过那样复杂度就高了，这里应该是只有 O(N) 的复杂度。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
<category>Java</category>
|
|
<category>Linked List</category>
|
|
<category>java</category>
|
|
<category>Linked List</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
<tag>Linked List</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 1862 向下取整数对和 ( Sum of Floored Pairs *Hard* ) 题解分析</title>
|
|
<url>/2022/09/11/Leetcode-1862-%E5%90%91%E4%B8%8B%E5%8F%96%E6%95%B4%E6%95%B0%E5%AF%B9%E5%92%8C-Sum-of-Floored-Pairs-Hard-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h3 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h3><p>Given an integer array <code>nums</code>, return the sum of <code>floor(nums[i] / nums[j])</code> for all pairs of indices <code>0 <= i, j < nums.length</code> in the array. Since the answer may be too large, return it modulo <code>10^9 + 7</code>.</p>
|
|
<p>The <code>floor()</code> function returns the integer part of the division.</p>
|
|
<p>对应中文<br>给你一个整数数组 <code>nums</code> ，请你返回所有下标对 <code>0 <= i, j < nums.length</code> 的 <code>floor(nums[i] / nums[j])</code> 结果之和。由于答案可能会很大，请你返回答案对<code>10^9 + 7</code> 取余 的结果。</p>
|
|
<p>函数 <code>floor()</code> 返回输入数字的整数部分。</p>
|
|
<h3 id="示例"><a href="#示例" class="headerlink" title="示例"></a>示例</h3><h4 id="Example-1"><a href="#Example-1" class="headerlink" title="Example 1:"></a>Example 1:</h4><blockquote>
|
|
<p><strong>Input:</strong> nums = [2,5,9]<br><strong>Output:</strong> 10<br><strong>Explanation:</strong><br>floor(2 / 5) = floor(2 / 9) = floor(5 / 9) = 0<br>floor(2 / 2) = floor(5 / 5) = floor(9 / 9) = 1<br>floor(5 / 2) = 2<br>floor(9 / 2) = 4<br>floor(9 / 5) = 1<br>We calculate the floor of the division for every pair of indices in the array then sum them up. </p>
|
|
</blockquote>
|
|
<h4 id="Example-2"><a href="#Example-2" class="headerlink" title="Example 2:"></a>Example 2:</h4><blockquote>
|
|
<p><strong>Input:</strong> nums = [7,7,7,7,7,7,7]<br><strong>Output:</strong> 49 </p>
|
|
</blockquote>
|
|
<h4 id="Constraints"><a href="#Constraints" class="headerlink" title="Constraints:"></a>Constraints:</h4><ul>
|
|
<li><code>1 <= nums.length <= 10^5</code></li>
|
|
<li><code>1 <= nums[i] <= 10^5</code></li>
|
|
</ul>
|
|
<h3 id="简析"><a href="#简析" class="headerlink" title="简析"></a>简析</h3><p>这题不愧是 hard，要不是看了讨论区的一个大神的解答感觉从头做得想好久，<br>主要是两点，对于任何一个在里面的数，随便举个例子是 k，最简单的就是循环所有数对 k 除一下，<br>这样效率会很低，那么对于 k 有什么规律呢，就是对于所有小于 k 的数，往下取整都是 0，所以不用考虑，<br>对于所有大于 k 的数我们可以分成一个个的区间，[k,2k-1),[2k,3k-1),[3k,4k-1)……对于这些区间的<br>除了 k 往下取整，每个区间内的都是一样的，所以可以简化为对于任意一个 k，我只要知道与k 相同的有多少个，然后比 k 大的各个区间各有多少个数就可以了 </p>
|
|
<h3 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">static</span> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">MAXE5</span> <span class="operator">=</span> <span class="number">100_000</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">static</span> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">MODULUSE9</span> <span class="operator">=</span> <span class="number">1_000_000_000</span> + <span class="number">7</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="type">int</span> <span class="title function_">sumOfFlooredPairs</span><span class="params">(<span class="type">int</span>[] nums)</span> {</span><br><span class="line"> <span class="type">int</span>[] counts = <span class="keyword">new</span> <span class="title class_">int</span>[MAXE5+<span class="number">1</span>];</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> num : nums) {</span><br><span class="line"> counts[num]++;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 这里就是很巧妙的给后一个加上前一个的值，这样其实前后任意两者之差就是这中间的元素数量</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">1</span>; i <= MAXE5; i++) {</span><br><span class="line"> counts[i] += counts[i - <span class="number">1</span>];</span><br><span class="line"> }</span><br><span class="line"> <span class="type">long</span> <span class="variable">total</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">1</span>; i <= MAXE5; i++) {</span><br><span class="line"> <span class="type">long</span> <span class="variable">sum</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">if</span> (counts[i] == counts[i-<span class="number">1</span>]) {</span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">j</span> <span class="operator">=</span> <span class="number">1</span>; i*j <= MAXE5; j++) {</span><br><span class="line"> <span class="type">int</span> <span class="variable">min</span> <span class="operator">=</span> i * j - <span class="number">1</span>;</span><br><span class="line"> <span class="type">int</span> <span class="variable">upper</span> <span class="operator">=</span> i * (j + <span class="number">1</span>) - <span class="number">1</span>;</span><br><span class="line"> <span class="comment">// 在每一个区间内的数量，</span></span><br><span class="line"> sum += (counts[Math.min(upper, MAXE5)] - counts[min]) * (<span class="type">long</span>)j;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 左边乘数的数量，即 i 位置的元素数量</span></span><br><span class="line"> total = (total + (sum % MODULUSE9 ) * (counts[i] - counts[i-<span class="number">1</span>])) % MODULUSE9;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> (<span class="type">int</span>)total;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>贴出来大神的解析,<a href="https://leetcode.com/problems/sum-of-floored-pairs/discuss/1210222/Java-O(n-log(n))-Straightforward-brute-force-with-explanation-76-ms">解析</a></p>
|
|
<h3 id="结果"><a href="#结果" class="headerlink" title="结果"></a>结果</h3><p><img data-src="https://img.nicksxs.me/uPic/U6MYqd.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 16 最接近的三数之和 ( 3Sum Closest *Medium* ) 题解分析</title>
|
|
<url>/2022/08/06/Leetcode-16-%E6%9C%80%E6%8E%A5%E8%BF%91%E7%9A%84%E4%B8%89%E6%95%B0%E4%B9%8B%E5%92%8C-3Sum-Closest-Medium-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h3 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h3><p>Given an integer array <code>nums</code> of length <code>n</code> and an integer <code>target</code>, find three integers in <code>nums</code> such that the sum is closest to <code>target</code>. </p>
|
|
<p>Return the sum of the three integers. </p>
|
|
<p>You may assume that each input would have exactly one solution. </p>
|
|
<p>简单解释下就是之前是要三数之和等于目标值，现在是找到最接近的三数之和。 </p>
|
|
<h3 id="示例"><a href="#示例" class="headerlink" title="示例"></a>示例</h3><h4 id="Example-1"><a href="#Example-1" class="headerlink" title="Example 1:"></a>Example 1:</h4><blockquote>
|
|
<p><strong>Input:</strong> nums = [-1,2,1,-4], target = 1<br><strong>Output:</strong> 2<br><strong>Explanation:</strong> The sum that is closest to the target is 2. (-1 + 2 + 1 = 2). </p>
|
|
</blockquote>
|
|
<h4 id="Example-2"><a href="#Example-2" class="headerlink" title="Example 2:"></a>Example 2:</h4><blockquote>
|
|
<p><strong>Input:</strong> nums = [0,0,0], target = 1<br><strong>Output:</strong> 0 </p>
|
|
</blockquote>
|
|
<h4 id="Constraints"><a href="#Constraints" class="headerlink" title="Constraints:"></a>Constraints:</h4><ul>
|
|
<li><code>3 <= nums.length <= 1000</code></li>
|
|
<li><code>-1000 <= nums[i] <= 1000</code></li>
|
|
<li><code>-10^4 <= target <= 10^4</code></li>
|
|
</ul>
|
|
<h3 id="简单解析"><a href="#简单解析" class="headerlink" title="简单解析"></a>简单解析</h3><p>这个题思路上来讲不难，也是用原来三数之和的方式去做，利用”双指针法”或者其它描述法，但是需要简化逻辑</p>
|
|
<h3 id="code"><a href="#code" class="headerlink" title="code"></a>code</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">int</span> <span class="title function_">threeSumClosest</span><span class="params">(<span class="type">int</span>[] nums, <span class="type">int</span> target)</span> {</span><br><span class="line"> Arrays.sort(nums);</span><br><span class="line"> <span class="comment">// 当前最近的和</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">closestSum</span> <span class="operator">=</span> nums[<span class="number">0</span>] + nums[<span class="number">1</span>] + nums[nums.length - <span class="number">1</span>];</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < nums.length - <span class="number">2</span>; i++) {</span><br><span class="line"> <span class="keyword">if</span> (i == <span class="number">0</span> || nums[i] != nums[i - <span class="number">1</span>]) {</span><br><span class="line"> <span class="comment">// 左指针</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">left</span> <span class="operator">=</span> i + <span class="number">1</span>;</span><br><span class="line"> <span class="comment">// 右指针</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">right</span> <span class="operator">=</span> nums.length - <span class="number">1</span>;</span><br><span class="line"> <span class="comment">// 判断是否遍历完了</span></span><br><span class="line"> <span class="keyword">while</span> (left < right) {</span><br><span class="line"> <span class="comment">// 当前的和</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">sum</span> <span class="operator">=</span> nums[i] + nums[left] + nums[right];</span><br><span class="line"> <span class="comment">// 小优化，相等就略过了</span></span><br><span class="line"> <span class="keyword">while</span> (left < right && nums[left] == nums[left + <span class="number">1</span>]) {</span><br><span class="line"> left++;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">while</span> (left < right && nums[right] == nums[right - <span class="number">1</span>]) {</span><br><span class="line"> right--;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 这里判断，其实也还是希望趋近目标值</span></span><br><span class="line"> <span class="keyword">if</span> (sum < target) {</span><br><span class="line"> left++;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> right--;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 判断是否需要替换</span></span><br><span class="line"> <span class="keyword">if</span> (Math.abs(sum - target) < Math.abs(closestSum - target)) {</span><br><span class="line"> closestSum = sum;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> closestSum;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<h3 id="结果"><a href="#结果" class="headerlink" title="结果"></a>结果</h3><p><img data-src="https://img.nicksxs.me/uPic/3TE3R7.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
<tag>3Sum Closest</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 20 有效的括号 ( Valid Parentheses *Easy* ) 题解分析</title>
|
|
<url>/2022/07/02/Leetcode-20-%E6%9C%89%E6%95%88%E7%9A%84%E6%8B%AC%E5%8F%B7-Valid-Parentheses-Easy-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h3 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h3><p>Given a string <code>s</code> containing just the characters <code>'('</code>, <code>')'</code>, <code>'{'</code>, <code>'}'</code>, <code>'['</code> and <code>']'</code>, determine if the input string is valid.</p>
|
|
<p>An input string is valid if:</p>
|
|
<ol>
|
|
<li>Open brackets must be closed by the same type of brackets.</li>
|
|
<li>Open brackets must be closed in the correct order.</li>
|
|
</ol>
|
|
<h3 id="示例"><a href="#示例" class="headerlink" title="示例"></a>示例</h3><h4 id="Example-1"><a href="#Example-1" class="headerlink" title="Example 1:"></a>Example 1:</h4><blockquote>
|
|
<p>Input: s = “()”<br>Output: true </p>
|
|
</blockquote>
|
|
<h4 id="Example-2"><a href="#Example-2" class="headerlink" title="Example 2:"></a>Example 2:</h4><blockquote>
|
|
<p>Input: s = “()[]{}”<br>Output: true </p>
|
|
</blockquote>
|
|
<h4 id="Example-3"><a href="#Example-3" class="headerlink" title="Example 3:"></a>Example 3:</h4><blockquote>
|
|
<p>Input: s = “(]”<br>Output: false </p>
|
|
</blockquote>
|
|
<h4 id="Constraints"><a href="#Constraints" class="headerlink" title="Constraints:"></a>Constraints:</h4><ul>
|
|
<li><code>1 <= s.length <= 10^4</code></li>
|
|
<li><code>s</code> consists of parentheses only <code>'()[]{}'</code>.</li>
|
|
</ul>
|
|
<h3 id="解析"><a href="#解析" class="headerlink" title="解析"></a>解析</h3><p>easy题，并且看起来也是比较简单的，三种括号按对匹配，直接用栈来做，栈里面存的是括号的类型，如果是左括号，就放入栈中，如果是右括号，就把栈顶的元素弹出，如果弹出的元素不是左括号，就返回false，如果弹出的元素是左括号，就继续往下走，如果遍历完了，如果栈里面还有元素，就返回false，如果遍历完了，如果栈里面没有元素，就返回true</p>
|
|
<h3 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">Solution</span> {</span><br><span class="line"> <span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">isValid</span><span class="params">(String s)</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (s.length() % <span class="number">2</span> != <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> Stack<String> stk = <span class="keyword">new</span> <span class="title class_">Stack</span><>();</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < s.length(); i++) {</span><br><span class="line"> <span class="keyword">if</span> (s.charAt(i) == <span class="string">'{'</span> || s.charAt(i) == <span class="string">'('</span> || s.charAt(i) == <span class="string">'['</span>) {</span><br><span class="line"> stk.push(String.valueOf(s.charAt(i)));</span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (s.charAt(i) == <span class="string">'}'</span>) {</span><br><span class="line"> <span class="keyword">if</span> (stk.isEmpty()) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">String</span> <span class="variable">cur</span> <span class="operator">=</span> stk.peek();</span><br><span class="line"> <span class="keyword">if</span> (cur.charAt(<span class="number">0</span>) != <span class="string">'{'</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> stk.pop();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (s.charAt(i) == <span class="string">']'</span>) {</span><br><span class="line"> <span class="keyword">if</span> (stk.isEmpty()) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">String</span> <span class="variable">cur</span> <span class="operator">=</span> stk.peek();</span><br><span class="line"> <span class="keyword">if</span> (cur.charAt(<span class="number">0</span>) != <span class="string">'['</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> stk.pop();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (s.charAt(i) == <span class="string">')'</span>) {</span><br><span class="line"> <span class="keyword">if</span> (stk.isEmpty()) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">String</span> <span class="variable">cur</span> <span class="operator">=</span> stk.peek();</span><br><span class="line"> <span class="keyword">if</span> (cur.charAt(<span class="number">0</span>) != <span class="string">'('</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> stk.pop();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> stk.size() == <span class="number">0</span>;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 236 二叉树的最近公共祖先(Lowest Common Ancestor of a Binary Tree) 题解分析</title>
|
|
<url>/2021/05/23/Leetcode-236-%E4%BA%8C%E5%8F%89%E6%A0%91%E7%9A%84%E6%9C%80%E8%BF%91%E5%85%AC%E5%85%B1%E7%A5%96%E5%85%88-Lowest-Common-Ancestor-of-a-Binary-Tree-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h3 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h3><p>Given a binary tree, find the lowest common ancestor (LCA) of two given nodes in the tree.</p>
|
|
<p>According to the <a href="https://en.wikipedia.org/wiki/Lowest_common_ancestor">definition of LCA on Wikipedia</a>: “The lowest common ancestor is defined between two nodes <code>p</code> and <code>q</code> as the lowest node in <code>T</code> that has both <code>p</code> and <code>q</code> as descendants (where we allow <strong>a node to be a descendant of itself</strong>).”</p>
|
|
<p>给定一个二叉树, 找到该树中两个指定节点的最近公共祖先。</p>
|
|
<p><a href="https://baike.baidu.com/item/%E6%9C%80%E8%BF%91%E5%85%AC%E5%85%B1%E7%A5%96%E5%85%88/8918834?fr=aladdin">百度百科</a>中最近公共祖先的定义为：“对于有根树 <code>T</code> 的两个节点 <code>p</code>、<code>q</code>，最近公共祖先表示为一个节点 <code>x</code>，满足 <code>x</code> 是 <code>p</code>、<code>q</code> 的祖先且 <code>x</code> 的深度尽可能大（<strong>一个节点也可以是它自己的祖先</strong>）。”</p>
|
|
<h2 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> TreeNode <span class="title function_">lowestCommonAncestor</span><span class="params">(TreeNode root, TreeNode p, TreeNode q)</span> {</span><br><span class="line"> <span class="comment">// 如果当前节点就是 p 或者是 q 的时候，就直接返回了</span></span><br><span class="line"> <span class="comment">// 当没找到，即 root == null 的时候也会返回 null，这是个重要的点</span></span><br><span class="line"> <span class="keyword">if</span> (root == <span class="literal">null</span> || root == p || root == q) <span class="keyword">return</span> root;</span><br><span class="line"> <span class="comment">// 在左子树中找 p 和 q</span></span><br><span class="line"> <span class="type">TreeNode</span> <span class="variable">left</span> <span class="operator">=</span> lowestCommonAncestor(root.left, p, q);</span><br><span class="line"> <span class="comment">// 在右子树中找 p 和 q</span></span><br><span class="line"> <span class="type">TreeNode</span> <span class="variable">right</span> <span class="operator">=</span> lowestCommonAncestor(root.right, p, q);</span><br><span class="line"> <span class="comment">// 当左边是 null 就直接返回右子树，但是这里不表示右边不是 null，所以这个顺序是不影响的</span></span><br><span class="line"> <span class="comment">// 考虑一种情况，如果一个节点的左右子树都是 null，那么其实对于这个节点来说首先两个子树分别调用</span></span><br><span class="line"> <span class="comment">// lowestCommonAncestor会在开头就返回 null，那么就是上面 left 跟 right 都是 null，然后走下面的判断的时候</span></span><br><span class="line"> <span class="comment">// 其实第一个 if 就返回了 null，如此递归返回就能达到当子树中没有找到 p 或者 q 的时候只返回 null</span></span><br><span class="line"> <span class="keyword">if</span> (left == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> right;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (right == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> left;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> root;</span><br><span class="line"> }</span><br><span class="line"><span class="comment">// if (right == null) {</span></span><br><span class="line"><span class="comment">// return left;</span></span><br><span class="line"><span class="comment">// } else if (left == null) {</span></span><br><span class="line"><span class="comment">// return right;</span></span><br><span class="line"><span class="comment">// } else {</span></span><br><span class="line"><span class="comment">// return root;</span></span><br><span class="line"><span class="comment">// }</span></span><br><span class="line"><span class="comment">// return left == null ? right : right == null ? left : root;</span></span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
<category>Lowest Common Ancestor of a Binary Tree</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
<tag>Lowest Common Ancestor of a Binary Tree</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 234 回文链表(Palindrome Linked List) 题解分析</title>
|
|
<url>/2020/11/15/Leetcode-234-%E5%9B%9E%E6%96%87%E8%81%94%E8%A1%A8-Palindrome-Linked-List-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h2 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h2><p>Given a singly linked list, determine if it is a palindrome.<br>给定一个单向链表，判断是否是回文链表</p>
|
|
<h3 id="例一-Example-1"><a href="#例一-Example-1" class="headerlink" title="例一 Example 1:"></a>例一 Example 1:</h3><p>Input: 1->2<br>Output: false</p>
|
|
<h3 id="例二-Example-2"><a href="#例二-Example-2" class="headerlink" title="例二 Example 2:"></a>例二 Example 2:</h3><p>Input: 1->2->2->1<br>Output: true</p>
|
|
<h3 id="挑战下自己"><a href="#挑战下自己" class="headerlink" title="挑战下自己"></a>挑战下自己</h3><p>Follow up:<br>Could you do it in O(n) time and O(1) space?</p>
|
|
<h2 id="简要分析"><a href="#简要分析" class="headerlink" title="简要分析"></a>简要分析</h2><p>首先这是个单向链表，如果是双向的就可以一个从头到尾，一个从尾到头，显然那样就没啥意思了，然后想过要不找到中点，然后用一个栈，把前一半塞进栈里，但是这种其实也比较麻烦，比如长度是奇偶数，然后如何找到中点，这倒是可以借助于双指针，还是比较麻烦，再想一想，回文链表，就跟最开始的一样，链表只有单向的，我用个栈不就可以逆向了么，先把链表整个塞进栈里，然后在一个个 pop 出来跟链表从头开始比较，全对上了就是回文了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Definition for singly-linked list.</span></span><br><span class="line"><span class="comment"> * public class ListNode {</span></span><br><span class="line"><span class="comment"> * int val;</span></span><br><span class="line"><span class="comment"> * ListNode next;</span></span><br><span class="line"><span class="comment"> * ListNode() {}</span></span><br><span class="line"><span class="comment"> * ListNode(int val) { this.val = val; }</span></span><br><span class="line"><span class="comment"> * ListNode(int val, ListNode next) { this.val = val; this.next = next; }</span></span><br><span class="line"><span class="comment"> * }</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">class</span> <span class="title class_">Solution</span> {</span><br><span class="line"> <span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">isPalindrome</span><span class="params">(ListNode head)</span> {</span><br><span class="line"> <span class="keyword">if</span> (head == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">ListNode</span> <span class="variable">tail</span> <span class="operator">=</span> head;</span><br><span class="line"> LinkedList<Integer> stack = <span class="keyword">new</span> <span class="title class_">LinkedList</span><>();</span><br><span class="line"> <span class="comment">// 这里就是一个循环，将所有元素依次压入栈</span></span><br><span class="line"> <span class="keyword">while</span> (tail != <span class="literal">null</span>) {</span><br><span class="line"> stack.push(tail.val);</span><br><span class="line"> tail = tail.next;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 在逐个 pop 出来，其实这个出来的顺序就等于链表从尾到头遍历，同时跟链表从头到尾遍历进行逐对对比</span></span><br><span class="line"> <span class="keyword">while</span> (!stack.isEmpty()) {</span><br><span class="line"> <span class="keyword">if</span> (stack.peekFirst() == head.val) {</span><br><span class="line"> stack.pollFirst();</span><br><span class="line"> head = head.next;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
<category>Java</category>
|
|
<category>Linked List</category>
|
|
<category>java</category>
|
|
<category>Linked List</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
<tag>Linked List</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 278 第一个错误的版本 ( First Bad Version *Easy* ) 题解分析</title>
|
|
<url>/2022/08/14/Leetcode-278-%E7%AC%AC%E4%B8%80%E4%B8%AA%E9%94%99%E8%AF%AF%E7%9A%84%E7%89%88%E6%9C%AC-First-Bad-Version-Easy-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h3 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h3><p>You are a product manager and currently leading a team to develop a new product. Unfortunately, the latest version of your product fails the quality check. Since each version is developed based on the previous version, all the versions after a bad version are also bad.</p>
|
|
<p>Suppose you have <code>n</code> versions <code>[1, 2, ..., n]</code> and you want to find out the first bad one, which causes all the following ones to be bad.</p>
|
|
<p>You are given an API <code>bool isBadVersion(version)</code> which returns whether <code>version</code> is bad. Implement a function to find the first bad version. You should minimize the number of calls to the API.</p>
|
|
<h3 id="示例"><a href="#示例" class="headerlink" title="示例"></a>示例</h3><h4 id="Example-1"><a href="#Example-1" class="headerlink" title="Example 1:"></a>Example 1:</h4><blockquote>
|
|
<p><strong>Input:</strong> n = 5, bad = 4<br><strong>Output:</strong> 4<br><strong>Explanation:</strong><br>call isBadVersion(3) -> false<br>call isBadVersion(5) -> true<br>call isBadVersion(4) -> true<br>Then 4 is the first bad version. </p>
|
|
</blockquote>
|
|
<h4 id="Example-2"><a href="#Example-2" class="headerlink" title="Example 2:"></a>Example 2:</h4><blockquote>
|
|
<p><strong>Input:</strong> n = 1, bad = 1<br><strong>Output:</strong> 1 </p>
|
|
</blockquote>
|
|
<h3 id="简析"><a href="#简析" class="headerlink" title="简析"></a>简析</h3><p>简单来说就是一个二分查找，但是这个问题其实处理起来还是需要搞清楚一些边界问题</p>
|
|
<h3 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">int</span> <span class="title function_">firstBadVersion</span><span class="params">(<span class="type">int</span> n)</span> {</span><br><span class="line"> <span class="comment">// 类似于双指针法</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">left</span> <span class="operator">=</span> <span class="number">1</span>, right = n, mid;</span><br><span class="line"> <span class="keyword">while</span> (left < right) {</span><br><span class="line"> <span class="comment">// 取中点</span></span><br><span class="line"> mid = left + (right - left) / <span class="number">2</span>;</span><br><span class="line"> <span class="comment">// 如果不是错误版本，就往右找</span></span><br><span class="line"> <span class="keyword">if</span> (!isBadVersion(mid)) {</span><br><span class="line"> left = mid + <span class="number">1</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// 如果是的话就往左查找</span></span><br><span class="line"> right = mid;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 这里考虑交界情况是，在上面循环中如果 left 是好的，right 是坏的，那进入循环的时候 mid == left</span></span><br><span class="line"> <span class="comment">// 然后 left = mid + 1 就会等于 right，循环条件就跳出了，此时 left 就是那个起始的错误点了</span></span><br><span class="line"> <span class="comment">// 其实这两个是同一个值</span></span><br><span class="line"> <span class="keyword">return</span> left;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>往右移动示例<br><img data-src="https://img.nicksxs.me/uPic/eWVkKN.png"><br>往左移动示例<br><img data-src="https://img.nicksxs.me/uPic/M5i8vr.png"></p>
|
|
<h3 id="结果"><a href="#结果" class="headerlink" title="结果"></a>结果</h3><p><img data-src="https://img.nicksxs.me/uPic/sXdyJz.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
<tag>First Bad Version</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 3 Longest Substring Without Repeating Characters 题解分析</title>
|
|
<url>/2020/09/20/Leetcode-3-Longest-Substring-Without-Repeating-Characters-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<p>又做了个题，看记录是以前用 C++写过的，现在捋一捋思路，用 Java 再写了一下，思路还是比较清晰的，但是边界细节处理得比较差</p>
|
|
<h2 id="简要介绍"><a href="#简要介绍" class="headerlink" title="简要介绍"></a>简要介绍</h2><p>Given a string <code>s</code>, find the length of the <strong>longest substring</strong> without repeating characters.</p>
|
|
<h2 id="样例"><a href="#样例" class="headerlink" title="样例"></a>样例</h2><h4 id="Example-1"><a href="#Example-1" class="headerlink" title="Example 1:"></a>Example 1:</h4><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Input: s = "abcabcbb"</span><br><span class="line">Output: 3</span><br><span class="line">Explanation: The answer is "abc", with the length of 3.</span><br></pre></td></tr></table></figure>
|
|
|
|
<h4 id="Example-2"><a href="#Example-2" class="headerlink" title="Example 2:"></a>Example 2:</h4><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Input: s = "bbbbb"</span><br><span class="line">Output: 1</span><br><span class="line">Explanation: The answer is "b", with the length of 1.</span><br></pre></td></tr></table></figure>
|
|
<h4 id="Example-3"><a href="#Example-3" class="headerlink" title="Example 3:"></a>Example 3:</h4><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Input: s = "pwwkew"</span><br><span class="line">Output: 3</span><br><span class="line">Explanation: The answer is "wke", with the length of 3.</span><br><span class="line">Notice that the answer must be a substring, "pwke" is a subsequence and not a substring.</span><br></pre></td></tr></table></figure>
|
|
<h4 id="Example-4"><a href="#Example-4" class="headerlink" title="Example 4:"></a>Example 4:</h4><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Input: s = ""</span><br><span class="line">Output: 0</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>就是一个最长不重复的字符串长度，因为也是中等难度的题，不太需要特别复杂的思考，最基本的就是O(N*N)两重循环，不过显然不太好，万一超时间，还有一种就是线性复杂度的了，这个就是需要搞定一个思路，比如字符串时 <code>a</code>bcdefg<code>a</code>qwrty，比如遍历到第二个<code>a</code>的时候其实不用再从头去遍历了，只要把前面那个<code>a</code>给排除掉，继续往下算就好了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">Solution</span> {</span><br><span class="line"> Map<String, Integer> counter = <span class="keyword">new</span> <span class="title class_">HashMap</span><>();</span><br><span class="line"> <span class="keyword">public</span> <span class="type">int</span> <span class="title function_">lengthOfLongestSubstring</span><span class="params">(String s)</span> {</span><br><span class="line"> <span class="type">int</span> <span class="variable">length</span> <span class="operator">=</span> s.length();</span><br><span class="line"> <span class="comment">// 当前的长度</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">subStringLength</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="comment">// 最长的长度</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">maxSubStringLength</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="comment">// 考虑到重复的位置已经被跳过的情况，即已经在当前长度的字符串范围之前的重复字符不需要回溯</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">lastDuplicatePos</span> <span class="operator">=</span> -<span class="number">1</span>;</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < length; i++) {</span><br><span class="line"> <span class="comment">// 使用 map 存储字符和上一次出现的位置，如果存在并且大于上一次重复位置</span></span><br><span class="line"> <span class="keyword">if</span> (counter.get(String.valueOf(s.charAt(i))) != <span class="literal">null</span> && counter.get(String.valueOf(s.charAt(i))) > lastDuplicatePos) {</span><br><span class="line"> <span class="comment">// 记录重复位置</span></span><br><span class="line"> lastDuplicatePos = counter.get(String.valueOf(s.charAt(i)));</span><br><span class="line"> <span class="comment">// 重置不重复子串的长度，减去重复起点</span></span><br><span class="line"> subStringLength = i - counter.get(String.valueOf(s.charAt(i))) - <span class="number">1</span>;</span><br><span class="line"> <span class="comment">// 替换当前位置</span></span><br><span class="line"> counter.replace(String.valueOf(s.charAt(i)), i);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// 如果不存在就直接 put</span></span><br><span class="line"> counter.put(String.valueOf(s.charAt(i)), i);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 长度累加</span></span><br><span class="line"> subStringLength++;</span><br><span class="line"> <span class="keyword">if</span> (subStringLength > maxSubStringLength) {</span><br><span class="line"> <span class="comment">// 简单替换</span></span><br><span class="line"> maxSubStringLength = subStringLength;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> maxSubStringLength;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>注释应该写的比较清楚了。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
<category>Java</category>
|
|
<category>字符串 - online</category>
|
|
<category>java</category>
|
|
<category>string</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
<tag>string</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Java 线程池系列-第二篇</title>
|
|
<url>/2024/02/18/Java-%E7%BA%BF%E7%A8%8B%E6%B1%A0%E7%B3%BB%E5%88%97-%E7%AC%AC%E4%BA%8C%E7%AF%87/</url>
|
|
<content><![CDATA[<p>介绍了线程池的目的和实现概述，以及如何初始化的，我们就来开始看看线程池最重要的执行过程<br>老规矩，还是先把注释翻译下，这个对理解逻辑其实非常重要，后面可以循着注释的逻辑来看代码<br>第一步，如果是少于核心线程数的线程正在运行，那么尝试去开启一个新线程，并把提交的命令<br>command 最为 first task，这里调用了 addWorker，会自动检查运行状态和线程数，并通过<br>返回 false 来提示不应该增加线程，比如已经有其他任务先创建了线程导致已经超过了核心线程数<br>第二步，在前面不符合的情况下，也就是线程数已经大于等于核心线程数了或者在调用 addWorker<br>的时候校验发现线程数已经大于等于核心线程数或者线程池运行状态不是正在运行了就会进入下一个判断<br>首先还是判断线程池是否正在运行，然后将 command 放进队列，如果成功进队了，还需要再次进行校验<br>如果线程池状态不是正在运行了，则需要再出队，然后执行拒绝策略，如果状态正常，但是线程被回收完了<br>那需要创建线程<br>第三步，如果不能正常进队列，会尝试再启动一个新线程，这里表示核心线程数满了，并且队列也满了，<br>则需要再开启一个新线程，如果开启失败则执行拒绝策略<br>代码注释基本已经把代码讲得很详细了 </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">execute</span><span class="params">(Runnable command)</span> {</span><br><span class="line"> <span class="keyword">if</span> (command == <span class="literal">null</span>)</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">NullPointerException</span>();</span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> * Proceed in 3 steps:</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * 1. If fewer than corePoolSize threads are running, try to</span></span><br><span class="line"><span class="comment"> * start a new thread with the given command as its first</span></span><br><span class="line"><span class="comment"> * task. The call to addWorker atomically checks runState and</span></span><br><span class="line"><span class="comment"> * workerCount, and so prevents false alarms that would add</span></span><br><span class="line"><span class="comment"> * threads when it shouldn't, by returning false.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * 2. If a task can be successfully queued, then we still need</span></span><br><span class="line"><span class="comment"> * to double-check whether we should have added a thread</span></span><br><span class="line"><span class="comment"> * (because existing ones died since last checking) or that</span></span><br><span class="line"><span class="comment"> * the pool shut down since entry into this method. So we</span></span><br><span class="line"><span class="comment"> * recheck state and if necessary roll back the enqueuing if</span></span><br><span class="line"><span class="comment"> * stopped, or start a new thread if there are none.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * 3. If we cannot queue task, then we try to add a new</span></span><br><span class="line"><span class="comment"> * thread. If it fails, we know we are shut down or saturated</span></span><br><span class="line"><span class="comment"> * and so reject the task.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">c</span> <span class="operator">=</span> ctl.get();</span><br><span class="line"> <span class="comment">// 这就是第一步</span></span><br><span class="line"> <span class="keyword">if</span> (workerCountOf(c) < corePoolSize) {</span><br><span class="line"> <span class="keyword">if</span> (addWorker(command, <span class="literal">true</span>))</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> <span class="comment">// 注意这里是第一步可能失败了，获取一下最新的状态</span></span><br><span class="line"> c = ctl.get();</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 这里是第二步</span></span><br><span class="line"> <span class="keyword">if</span> (isRunning(c) && workQueue.offer(command)) {</span><br><span class="line"> <span class="comment">// 每次都获取最新的状态</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">recheck</span> <span class="operator">=</span> ctl.get();</span><br><span class="line"> <span class="keyword">if</span> (! isRunning(recheck) && remove(command))</span><br><span class="line"> reject(command);</span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span> (workerCountOf(recheck) == <span class="number">0</span>)</span><br><span class="line"> <span class="comment">// 这里是判断如果一个线程都没了的话，注意添加的是非核心线程</span></span><br><span class="line"> addWorker(<span class="literal">null</span>, <span class="literal">false</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 这里第三步是添加线程，并且是非核心线程</span></span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span> (!addWorker(command, <span class="literal">false</span>))</span><br><span class="line"> reject(command);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里面比较重要的就是 addWorker 方法，我们也来看一下<br>这里用到了 <code>break retry; </code> 就是在循环中跳出到这个 retry 的位置继续执行<br>然后还是获取运行状态，如果是非运行状态，<br>并且<code>rs == SHUTDOWN</code> 与 <code>firstTask == null</code> 和 <code>! workQueue.isEmpty()</code>，至少一个为false<br>因为只有当 SHUTDOWN 状态才会出现后面两者为 false 的情况，如果是 SHUTDOWN ，队列不为空还是需要执行完队列里的任务<br>然后是判断线程数量与传入参数 core 还有核心线程数跟最大线程数的对比，如果是超过了就返回 false 也就是外层的第一步的内部<br>if 就会继续执行后面的获取线程池状态，然后是 cas 增加线程数量，如果失败则 break 跳到方法开头，如果成功则继续重新获取 c<br>判断线程池运行状态，如果不一致则从外层继续进入<br>后面的就是新建一个 worker，然后获取锁，继续判断线程池状态，如果在运行中或者队列不为空（隐含条件）则继续执行添加 worker<br>并判断是否需要更新历史最大线程数，更改线程添加状态，然后启动线程，标记线程启动状态 </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="type">boolean</span> <span class="title function_">addWorker</span><span class="params">(Runnable firstTask, <span class="type">boolean</span> core)</span> {</span><br><span class="line"> retry:</span><br><span class="line"> <span class="keyword">for</span> (;;) {</span><br><span class="line"> <span class="type">int</span> <span class="variable">c</span> <span class="operator">=</span> ctl.get();</span><br><span class="line"> <span class="type">int</span> <span class="variable">rs</span> <span class="operator">=</span> runStateOf(c);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Check if queue empty only if necessary.</span></span><br><span class="line"> <span class="keyword">if</span> (rs >= SHUTDOWN && ! (rs == SHUTDOWN && firstTask == <span class="literal">null</span> && ! workQueue.isEmpty()))</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (;;) {</span><br><span class="line"> <span class="type">int</span> <span class="variable">wc</span> <span class="operator">=</span> workerCountOf(c);</span><br><span class="line"> <span class="keyword">if</span> (wc >= CAPACITY ||</span><br><span class="line"> wc >= (core ? corePoolSize : maximumPoolSize))</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="keyword">if</span> (compareAndIncrementWorkerCount(c))</span><br><span class="line"> <span class="keyword">break</span> retry;</span><br><span class="line"> c = ctl.get(); <span class="comment">// Re-read ctl</span></span><br><span class="line"> <span class="keyword">if</span> (runStateOf(c) != rs)</span><br><span class="line"> <span class="keyword">continue</span> retry;</span><br><span class="line"> <span class="comment">// else CAS failed due to workerCount change; retry inner loop</span></span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">boolean</span> <span class="variable">workerStarted</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">workerAdded</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="type">Worker</span> <span class="variable">w</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> w = <span class="keyword">new</span> <span class="title class_">Worker</span>(firstTask);</span><br><span class="line"> <span class="keyword">final</span> <span class="type">Thread</span> <span class="variable">t</span> <span class="operator">=</span> w.thread;</span><br><span class="line"> <span class="keyword">if</span> (t != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">ReentrantLock</span> <span class="variable">mainLock</span> <span class="operator">=</span> <span class="built_in">this</span>.mainLock;</span><br><span class="line"> mainLock.lock();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// Recheck while holding lock.</span></span><br><span class="line"> <span class="comment">// Back out on ThreadFactory failure or if</span></span><br><span class="line"> <span class="comment">// shut down before lock acquired.</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">rs</span> <span class="operator">=</span> runStateOf(ctl.get());</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (rs < SHUTDOWN ||</span><br><span class="line"> (rs == SHUTDOWN && firstTask == <span class="literal">null</span>)) {</span><br><span class="line"> <span class="keyword">if</span> (t.isAlive()) <span class="comment">// precheck that t is startable</span></span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalThreadStateException</span>();</span><br><span class="line"> workers.add(w);</span><br><span class="line"> <span class="type">int</span> <span class="variable">s</span> <span class="operator">=</span> workers.size();</span><br><span class="line"> <span class="keyword">if</span> (s > largestPoolSize)</span><br><span class="line"> largestPoolSize = s;</span><br><span class="line"> workerAdded = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> mainLock.unlock();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (workerAdded) {</span><br><span class="line"> t.start();</span><br><span class="line"> workerStarted = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="keyword">if</span> (! workerStarted)</span><br><span class="line"> addWorkerFailed(w);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> workerStarted;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>后面会继续讲下线程如何执行队列里的任务</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 349 两个数组的交集 ( Intersection of Two Arrays *Easy* ) 题解分析</title>
|
|
<url>/2022/03/07/Leetcode-349-%E4%B8%A4%E4%B8%AA%E6%95%B0%E7%BB%84%E7%9A%84%E4%BA%A4%E9%9B%86-Intersection-of-Two-Arrays-Easy-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h3 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h3><p>给定两个数组 <code>nums1</code> 和 <code>nums2</code> ，返回 它们的交集 。输出结果中的每个元素一定是 <strong>唯一</strong> 的。我们可以 <strong>不考虑输出结果的顺序</strong> 。 </p>
|
|
<p> </p>
|
|
<h3 id="示例"><a href="#示例" class="headerlink" title="示例"></a>示例</h3><blockquote>
|
|
<p>示例 1：</p>
|
|
<p><strong>输入</strong>：nums1 = [1,2,2,1], nums2 = [2,2]<br><strong>输出</strong>：[2] </p>
|
|
</blockquote>
|
|
<blockquote>
|
|
<p>示例 2：</p>
|
|
<p><strong>输入</strong>：nums1 = [4,9,5], nums2 = [9,4,9,8,4]<br><strong>输出</strong>：[9,4]<br><strong>解释</strong>：[4,9] 也是可通过的
|
|
</p>
|
|
</blockquote>
|
|
<h3 id="提示："><a href="#提示：" class="headerlink" title="提示："></a>提示：</h3><ul>
|
|
<li><code>1 <= nums1.length, nums2.length <= 1000</code></li>
|
|
<li><code>0 <= nums1[i], nums2[i] <= 1000</code></li>
|
|
</ul>
|
|
<h3 id="分析与题解"><a href="#分析与题解" class="headerlink" title="分析与题解"></a>分析与题解</h3><p>两个数组的交集，最简单就是两层循环了把两个都存在的找出来，不过还有个要去重的问题，稍微思考下可以使用集合 <code>set</code> 来处理，先把一个数组全丢进去，再对比另外一个，如果出现在第一个集合里就丢进一个新的集合，最后转换成数组，这次我稍微取了个巧，因为看到了提示里的条件，两个数组中的元素都是不大于 1000 的，所以就搞了个 1000 长度的数组，如果在第一个数组出现，就在对应的下标设置成 1，如果在第二个数组也出现了就加 1，</p>
|
|
<h4 id="code"><a href="#code" class="headerlink" title="code"></a>code</h4><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">int</span>[] intersection(<span class="type">int</span>[] nums1, <span class="type">int</span>[] nums2) {</span><br><span class="line"> <span class="comment">// 大小是 1000 的数组，如果没有提示的条件就没法这么做</span></span><br><span class="line"> <span class="comment">// define a array which size is 1000, and can not be done like this without the condition in notice</span></span><br><span class="line"> <span class="type">int</span>[] inter = <span class="keyword">new</span> <span class="title class_">int</span>[<span class="number">1000</span>];</span><br><span class="line"> <span class="type">int</span>[] outer;</span><br><span class="line"> <span class="type">int</span> <span class="variable">m</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> j : nums1) {</span><br><span class="line"> <span class="comment">// 这里得是设置成 1，因为有可能 nums1 就出现了重复元素，如果直接++会造成结果重复</span></span><br><span class="line"> <span class="comment">// need to be set 1, cause element in nums1 can be duplicated</span></span><br><span class="line"> inter[j] = <span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> j : nums2) {</span><br><span class="line"> <span class="keyword">if</span> (inter[j] > <span class="number">0</span>) {</span><br><span class="line"> <span class="comment">// 这里可以直接+1，因为后面判断只需要判断大于 1</span></span><br><span class="line"> <span class="comment">// just plus 1, cause we can judge with condition that larger than 1</span></span><br><span class="line"> inter[j] += <span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < inter.length; i++) {</span><br><span class="line"> <span class="comment">// 统计下元素数量</span></span><br><span class="line"> <span class="comment">// count distinct elements</span></span><br><span class="line"> <span class="keyword">if</span> (inter[i] > <span class="number">1</span>) {</span><br><span class="line"> m++;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// initial a array of size m</span></span><br><span class="line"> outer = <span class="keyword">new</span> <span class="title class_">int</span>[m];</span><br><span class="line"> m = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < inter.length; i++) {</span><br><span class="line"> <span class="keyword">if</span> (inter[i] > <span class="number">1</span>) {</span><br><span class="line"> <span class="comment">// add to outer</span></span><br><span class="line"> outer[m++] = i;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> outer;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
<tag>Intersection of Two Arrays</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 25 Reverse Nodes in k-Group 题解分析</title>
|
|
<url>/2020/08/16/Leetcode-25-Reverse-Nodes-in-k-Group-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90-1/</url>
|
|
<content><![CDATA[<p>Given a linked list, reverse the nodes of a linked list k at a time and return its modified list.</p>
|
|
<p>k is a positive integer and is less than or equal to the length of the linked list. If the number of nodes is not a multiple of k then left-out nodes in the end should remain as it is.</p>
|
|
<h3 id="Example"><a href="#Example" class="headerlink" title="Example:"></a>Example:</h3><p>Given this linked list: <code>1->2->3->4->5</code></p>
|
|
<p>For k = 2, you should return: <code>2->1->4->3->5</code></p>
|
|
<p>For k = 3, you should return: <code>3->2->1->4->5</code></p>
|
|
<h3 id="Note"><a href="#Note" class="headerlink" title="Note:"></a>Note:</h3><ul>
|
|
<li>Only constant extra memory is allowed.</li>
|
|
<li>You may not alter the values in the list’s nodes, only nodes itself may be changed.</li>
|
|
</ul>
|
|
<p>这个题也算是经典中的经典了，各种算法题中的保留曲目，可能不全一样，不过反正都是逆转，这个算是比较难度大的，普通的应该就是整个链表反转，或者只对一个链表中的 m 到 n 位做翻转，这里是 k 个一组做翻转，其实这道题比较难的点是两个，一个是想清楚怎么处理，一个是代码的处理<br>简单来看下</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> ListNode <span class="title function_">reverseKGroup</span><span class="params">(ListNode head, <span class="type">int</span> k)</span> {</span><br><span class="line"> <span class="keyword">if</span> (k <= <span class="number">1</span>) {</span><br><span class="line"> <span class="keyword">return</span> head;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (head == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">ListNode</span> <span class="variable">tempEnd</span> <span class="operator">=</span> head;</span><br><span class="line"> <span class="type">ListNode</span> <span class="variable">tempHead</span> <span class="operator">=</span> head;</span><br><span class="line"> <span class="type">ListNode</span> <span class="variable">lastHead</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">isHeadSet</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="keyword">while</span> (<span class="literal">true</span>) {</span><br><span class="line"> <span class="comment">// k 个一组获取头尾</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < k - <span class="number">1</span>; i++) {</span><br><span class="line"> System.out.println(tempEnd.val);</span><br><span class="line"> <span class="keyword">if</span> (tempEnd.next != <span class="literal">null</span>) {</span><br><span class="line"> tempEnd = tempEnd.next;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> tempEnd = <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (tempEnd != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">if</span> (!isHeadSet) {</span><br><span class="line"> <span class="comment">// 只有在第一组返回的时候需要赋值给 head</span></span><br><span class="line"> head = reverse(<span class="literal">null</span>, tempHead, tempEnd);</span><br><span class="line"> isHeadSet = !isHeadSet;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// 后面的需要将前一组的尾巴传进去，作为后一组头</span></span><br><span class="line"> reverse(lastHead, tempHead, tempEnd);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (tempHead.next != <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// 如果后续节点还有</span></span><br><span class="line"> lastHead = tempHead;</span><br><span class="line"> tempEnd = tempHead.next;</span><br><span class="line"> tempHead = tempHead.next;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> head;</span><br><span class="line">}</span><br><span class="line"><span class="keyword">public</span> ListNode <span class="title function_">reverse</span><span class="params">(ListNode lastHead, ListNode tempHead, ListNode tempEnd)</span> {</span><br><span class="line"> <span class="comment">// k 个一组的头尾传进来</span></span><br><span class="line"> <span class="type">ListNode</span> <span class="variable">tail</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">while</span> (tail != tempEnd) {</span><br><span class="line"> <span class="comment">// tail 表示是将头往后换时，把后面那个先拿着</span></span><br><span class="line"> tail = tempHead.next;</span><br><span class="line"> <span class="comment">// 然后就是交换</span></span><br><span class="line"> tempHead.next = tempEnd.next;</span><br><span class="line"> tempEnd.next = tempHead;</span><br><span class="line"> tempHead = tail;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (lastHead !=<span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// 如果不是第一组，则需要接上前一组的尾巴</span></span><br><span class="line"> lastHead.next = tail;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> tail;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 25 Reverse Nodes in k-Group 题解分析-再解分析</title>
|
|
<url>/2024/01/21/Leetcode-25-Reverse-Nodes-in-k-Group-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<p>上一次主要是给了一个解题方案，没有具体讲解，这次又做到了就来看下几种方案，链表转置一直是我比较疑惑的问题，特别是边界处理，而这个问题要把难度加大了<br>我先讲一下我一开始的思路和解题方法，首先就是写一个转置方法，就处理 k 个一组的内部转置，然后外部循环处理分组以及前后连接等问题，但是这里就涉及到一个问题，就是前后连接的话对于整个链表头，也就是第一个 k 元素组来说，头是个空的，就需要额外处理，一开始就是用判空做额外处理，然后在前后连接的时候也有一些困扰，看了自己的代码库发现其实之前也做过，而且发现这个思路跟以前做的还是一样的，只不过在处理 k 个元素内部的转置的时候有了点差异<br><img data-src="https://img.nicksxs.me/blog/0X0FuX-2024-01-21.9.11.15.png"><br>一开始的思路是这样的，想了下其实还是比较有问题，从处理难度上来说，在 k 组内处理的时候一开始 A 节点会是悬空的，然后得处理到组内最后一个元素的时候再接上，逻辑会更复杂，而另一种思路就是直接把 A 先挪到 C 后面，这样每次只需要移动一个，可能思路上会更清晰一点<br><img data-src="https://img.nicksxs.me/blog/0X0FuX.png"><br>也就是这样的，这种方案就是之前发过的题解代码，<a href="/2020/08/16/Leetcode-25-Reverse-Nodes-in-k-Group-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90-1/">上一篇</a></p>
|
|
<p>而最后这种则代码会简单一些，但是需要一定的理解成本，比较重要的一点是我们加了个虚拟的头结点，第一步会先获取下链表的总长度，然后在内循环里处理转置，重点就是分四步，<br>也就是图里的，第一步先把 cur 的下一个节点设置成 next 的 next 节点，这里就是图里 A 连接到 C，第二步是把 next 的 next 节点设置成虚拟头的 next节点，第三步是把 pre 的next 节点设置成 next，第四步是 next 往后移动</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">cur.next = next.next;</span><br><span class="line">next.next = pre.next;</span><br><span class="line">pre.next = next;</span><br><span class="line">next = cur.next;</span><br></pre></td></tr></table></figure>
|
|
|
|
<p><img data-src="https://img.nicksxs.me/blog/7uZX4o.png"><br>在备注下代码</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 4 寻找两个正序数组的中位数 ( Median of Two Sorted Arrays *Hard* ) 题解分析</title>
|
|
<url>/2022/03/27/Leetcode-4-%E5%AF%BB%E6%89%BE%E4%B8%A4%E4%B8%AA%E6%AD%A3%E5%BA%8F%E6%95%B0%E7%BB%84%E7%9A%84%E4%B8%AD%E4%BD%8D%E6%95%B0-Median-of-Two-Sorted-Arrays-Hard-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h3 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h3><p>给定两个大小分别为 <code>m</code> 和 <code>n</code> 的正序（从小到大）数组 <code>nums1</code> 和 <code>nums2</code>。请你找出并返回这两个正序数组的 <strong>中位数</strong> 。</p>
|
|
<p>算法的时间复杂度应该为 <code>O(log (m+n))</code> 。 </p>
|
|
<h4 id="示例-1："><a href="#示例-1：" class="headerlink" title="示例 1："></a>示例 1：</h4><blockquote>
|
|
<p>输入：nums1 = [1,3], nums2 = [2]<br>输出：2.00000<br>解释：合并数组 = [1,2,3] ，中位数 2 </p>
|
|
</blockquote>
|
|
<h4 id="示例-2："><a href="#示例-2：" class="headerlink" title="示例 2："></a>示例 2：</h4><blockquote>
|
|
<p>输入：nums1 = [1,2], nums2 = [3,4]<br>输出：2.50000<br>解释：合并数组 = [1,2,3,4] ，中位数 (2 + 3) / 2 = 2.5 </p>
|
|
</blockquote>
|
|
<h3 id="分析与题解"><a href="#分析与题解" class="headerlink" title="分析与题解"></a>分析与题解</h3><p>这个题也是我随机出来的，之前都是随机到 easy 的，而且是序号这么靠前的，然后翻一下，之前应该是用 C++做过的，具体的方法其实可以从他的算法时间复杂度要求看出来，大概率是要二分法这种，后面就结合代码来讲了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">double</span> <span class="title function_">findMedianSortedArrays</span><span class="params">(<span class="type">int</span>[] nums1, <span class="type">int</span>[] nums2)</span> {</span><br><span class="line"> <span class="type">int</span> <span class="variable">n1</span> <span class="operator">=</span> nums1.length;</span><br><span class="line"> <span class="type">int</span> <span class="variable">n2</span> <span class="operator">=</span> nums2.length;</span><br><span class="line"> <span class="keyword">if</span> (n1 > n2) {</span><br><span class="line"> <span class="keyword">return</span> findMedianSortedArrays(nums2, nums1);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 找到两个数组的中点下标</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">k</span> <span class="operator">=</span> (n1 + n2 + <span class="number">1</span> ) / <span class="number">2</span>;</span><br><span class="line"> <span class="comment">// 使用一个类似于二分法的查找方法</span></span><br><span class="line"> <span class="comment">// 起始值就是 num1 的头跟尾</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">left</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="type">int</span> <span class="variable">right</span> <span class="operator">=</span> n1;</span><br><span class="line"> <span class="keyword">while</span> (left < right) {</span><br><span class="line"> <span class="comment">// m1 表示我取的是 nums1 的中点，即二分法的方式</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">m1</span> <span class="operator">=</span> left + (right - left) / <span class="number">2</span>;</span><br><span class="line"> <span class="comment">// *** 这里是重点，因为这个问题也可以转换成找成 n1 + n2 那么多个数中的前 (n1 + n2 + 1) / 2 个</span></span><br><span class="line"> <span class="comment">// *** 因为两个数组都是排好序的，那么我从 num1 中取了 m1 个，从 num2 中就是去 k - m1 个</span></span><br><span class="line"> <span class="comment">// *** 但是不知道取出来大小是否正好是整体排序的第 (n1 + n2 + 1) / 2 个，所以需要二分法上下对比</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">m2</span> <span class="operator">=</span> k - m1;</span><br><span class="line"> <span class="comment">// 如果 nums1[m1] 小，那我在第一个数组 nums1 的二分查找就要把左端点改成前一次的中点 + 1 (不然就进死循环了</span></span><br><span class="line"> <span class="keyword">if</span> (nums1[m1] < nums2[m2 - <span class="number">1</span>]) {</span><br><span class="line"> left = m1 + <span class="number">1</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> right = m1;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 因为对比后其实我们只是拿到了一个位置，具体哪个是第 k 个就需要继续判断</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">m1</span> <span class="operator">=</span> left;</span><br><span class="line"> <span class="type">int</span> <span class="variable">m2</span> <span class="operator">=</span> k - left;</span><br><span class="line"> <span class="comment">// 如果 m1 或者 m2 有小于等于 0 的，那这个值可以先抛弃</span></span><br><span class="line"> <span class="comment">// m1 如果等于 0，就是 num1[0] 都比 nums2 中所有值都要大</span></span><br><span class="line"> <span class="comment">// m2 等于 0 的话 刚好相反</span></span><br><span class="line"> <span class="comment">// 可以这么推断，当其中一个是 0 的时候那么另一个 mx 值肯定是> 0 的，那么就是取的对应的这个下标的值</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">c1</span> <span class="operator">=</span> Math.max( m1 <= <span class="number">0</span> ? Integer.MIN_VALUE : nums1[m1 - <span class="number">1</span>] , m2 <= <span class="number">0</span> ? Integer.MIN_VALUE : nums2[m2 - <span class="number">1</span>]);</span><br><span class="line"> <span class="comment">// 如果两个数组的元素数量和是奇数，那就直接可以返回了，因为 m1 + m2 就是 k, 如果是一个数组，那这个元素其实就是 nums[k - 1]</span></span><br><span class="line"> <span class="comment">// 如果 m1 或者 m2 是 0，那另一个就是 k，取 mx - 1的下标就等于是 k - 1</span></span><br><span class="line"> <span class="comment">// 如果都不是 0，那就是取的了 nums1[m1 - 1] 与 nums2[m2 - 1]中的较大者，如果取得是后者，那么也就是 m1 + m2 - 1 的下标就是 k - 1</span></span><br><span class="line"> <span class="keyword">if</span> ((n1 + n2) % <span class="number">2</span> == <span class="number">1</span>) {</span><br><span class="line"> <span class="keyword">return</span> c1;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 如果是偶数个，那还要取两个数组后面的较小者，然后求平均值</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">c2</span> <span class="operator">=</span> Math.min(m1 >= n1 ? Integer.MAX_VALUE : nums1[m1], m2 >= n2 ? Integer.MAX_VALUE : nums2[m2]);</span><br><span class="line"> <span class="keyword">return</span> (c1 + c2) / <span class="number">2.0</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>前面考虑的方法还是比较繁琐，考虑了两个数组的各种交叉情况，后面这个参考了一些网上的解法，代码比较简洁，但是可能不容易一下子就搞明白，所以配合了比较多的注释。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
<tag>Median of Two Sorted Arrays</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 42 接雨水 (Trapping Rain Water) 题解分析</title>
|
|
<url>/2021/07/04/Leetcode-42-%E6%8E%A5%E9%9B%A8%E6%B0%B4-Trapping-Rain-Water-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h2 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h2><p>给定 n 个非负整数表示每个宽度为 1 的柱子的高度图，计算按此排列的柱子，下雨之后能接多少雨水。</p>
|
|
<h2 id="示例"><a href="#示例" class="headerlink" title="示例"></a>示例</h2><p><img data-src="https://img.nicksxs.me/uPic/AggmPZ.jpg"><br><strong>输入</strong>：height = [0,1,0,2,1,0,1,3,2,1,2,1]<br><strong>输出</strong>：6<br><strong>解释</strong>：上面是由数组 [0,1,0,2,1,0,1,3,2,1,2,1] 表示的高度图，在这种情况下，可以接 6 个单位的雨水（蓝色部分表示雨水）。</p>
|
|
<h2 id="简单分析"><a href="#简单分析" class="headerlink" title="简单分析"></a>简单分析</h2><p>其实最开始的想法是从左到右扫区间，就是示例中的第一个水槽跟第二个水槽都可以用这个办法解决<br><img data-src="https://img.nicksxs.me/uPic/zCLMbU.png"><br>前面这种是属于右侧比左侧高的情况，对于左侧高右侧低的就不行了，（写这篇的时候想起来可以再反着扫一遍可能可以）<br><img data-src="https://img.nicksxs.me/uPic/tO4tWh.png"><br>所以这个方案不好，贴一下这个方案的代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">int</span> <span class="title function_">trap</span><span class="params">(<span class="type">int</span>[] height)</span> {</span><br><span class="line"> <span class="type">int</span> <span class="variable">lastLeft</span> <span class="operator">=</span> -<span class="number">1</span>;</span><br><span class="line"> <span class="type">int</span> <span class="variable">sum</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="type">int</span> <span class="variable">tempSum</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">startFlag</span> <span class="operator">=</span> <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> j : height) {</span><br><span class="line"> <span class="keyword">if</span> (startFlag && j <= <span class="number">0</span>) {</span><br><span class="line"> startFlag = <span class="literal">false</span>;</span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (j >= lastLeft) {</span><br><span class="line"> sum += tempSum;</span><br><span class="line"> tempSum = <span class="number">0</span>;</span><br><span class="line"> lastLeft = j;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> tempSum += lastLeft - j;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> sum;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>后面结合网上的解法，其实可以反过来，对于每个格子找左右侧的最大值，取小的那个和当前格子的差值就是这一个的储水量了<br><img data-src="https://img.nicksxs.me/uPic/PXGiCa.png"><br>理解了这种想法，代码其实就不难了</p>
|
|
<h2 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">int</span> <span class="variable">n</span> <span class="operator">=</span> height.length;</span><br><span class="line"><span class="keyword">if</span> (n <= <span class="number">2</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br><span class="line"><span class="comment">// 思路转变下，其实可以对于每一格算储水量，算法就是找到这一格左边的最高点跟这一格右边的最高点，</span></span><br><span class="line"><span class="comment">// 比较两侧的最高点，取小的那个，然后再跟当前格子的高度对比，差值就是当前格的储水量</span></span><br><span class="line"><span class="type">int</span> maxL[] = <span class="keyword">new</span> <span class="title class_">int</span>[n];</span><br><span class="line"><span class="type">int</span> maxR[] = <span class="keyword">new</span> <span class="title class_">int</span>[n];</span><br><span class="line"><span class="type">int</span> <span class="variable">max</span> <span class="operator">=</span> height[<span class="number">0</span>];</span><br><span class="line">maxL[<span class="number">0</span>] = <span class="number">0</span>;</span><br><span class="line"><span class="comment">// 计算左侧的最高点</span></span><br><span class="line"><span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">1</span>; i < n - <span class="number">1</span>; i++) {</span><br><span class="line"> maxL[i] = max;</span><br><span class="line"> <span class="keyword">if</span> (max < height[i]) {</span><br><span class="line"> max = height[i];</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line">max = height[n - <span class="number">1</span>];</span><br><span class="line">maxR[n - <span class="number">1</span>] = <span class="number">0</span>;</span><br><span class="line"><span class="type">int</span> tempSum, sum = <span class="number">0</span>;</span><br><span class="line"><span class="comment">// 计算右侧的最高点，并且同步算出来储水量，节省一个循环</span></span><br><span class="line"><span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> n - <span class="number">2</span>; i > <span class="number">0</span>; i--) {</span><br><span class="line"> maxR[i] = max;</span><br><span class="line"> <span class="keyword">if</span> (height[i] > max) {</span><br><span class="line"> max = height[i];</span><br><span class="line"> }</span><br><span class="line"> tempSum = Math.min(maxL[i], maxR[i]) - height[i];</span><br><span class="line"> <span class="keyword">if</span> (tempSum > <span class="number">0</span>) {</span><br><span class="line"> sum += tempSum;</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"><span class="keyword">return</span> sum;</span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>dp</tag>
|
|
<tag>代码题解</tag>
|
|
<tag>Trapping Rain Water</tag>
|
|
<tag>接雨水</tag>
|
|
<tag>Leetcode 42</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 2 Add Two Numbers 题解分析</title>
|
|
<url>/2020/10/11/Leetcode-2-Add-Two-Numbers-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<p>又 roll 到了一个以前做过的题，不过现在用 Java 也来写一下，是 easy 级别的，所以就简单说下</p>
|
|
<h2 id="简要介绍"><a href="#简要介绍" class="headerlink" title="简要介绍"></a>简要介绍</h2><p>You are given two non-empty linked lists representing two non-negative integers. The digits are stored in reverse order, and each of their nodes contains a single digit. Add the two numbers and return the sum as a linked list.</p>
|
|
<p>You may assume the two numbers do not contain any leading zero, except the number 0 itself.<br>就是给了两个链表，用来表示两个非负的整数，在链表中倒序放着，每个节点包含一位的数字，把他们加起来以后也按照原来的链表结构输出</p>
|
|
<h2 id="样例"><a href="#样例" class="headerlink" title="样例"></a>样例</h2><h4 id="example-1"><a href="#example-1" class="headerlink" title="example 1"></a>example 1</h4><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Input: l1 = [2,4,3], l2 = [5,6,4]</span><br><span class="line">Output: [7,0,8]</span><br><span class="line">Explanation: 342 + 465 = 807.</span><br></pre></td></tr></table></figure>
|
|
|
|
<h4 id="example-2"><a href="#example-2" class="headerlink" title="example 2"></a>example 2</h4><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Input: l1 = [0], l2 = [0]</span><br><span class="line">Output: [0]</span><br></pre></td></tr></table></figure>
|
|
|
|
<h4 id="example-3"><a href="#example-3" class="headerlink" title="example 3"></a>example 3</h4><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Input: l1 = [9,9,9,9,9,9,9], l2 = [9,9,9,9]</span><br><span class="line">Output: [8,9,9,9,0,0,0,1]</span><br></pre></td></tr></table></figure>
|
|
|
|
<h2 id="题解"><a href="#题解" class="headerlink" title="题解"></a>题解</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> ListNode <span class="title function_">addTwoNumbers</span><span class="params">(ListNode l1, ListNode l2)</span> {</span><br><span class="line"> <span class="type">ListNode</span> <span class="variable">root</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ListNode</span>();</span><br><span class="line"> <span class="keyword">if</span> (l1 == <span class="literal">null</span> && l2 == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> root;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">ListNode</span> <span class="variable">tail</span> <span class="operator">=</span> root;</span><br><span class="line"> <span class="type">int</span> <span class="variable">entered</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="comment">// 这个条件加了 entered，就是还有进位的数</span></span><br><span class="line"> <span class="keyword">while</span> (l1 != <span class="literal">null</span> || l2 != <span class="literal">null</span> || entered != <span class="number">0</span>) {</span><br><span class="line"> <span class="type">int</span> <span class="variable">temp</span> <span class="operator">=</span> entered;</span><br><span class="line"> <span class="keyword">if</span> (l1 != <span class="literal">null</span>) {</span><br><span class="line"> temp += l1.val;</span><br><span class="line"> l1 = l1.next;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (l2 != <span class="literal">null</span>) {</span><br><span class="line"> temp += l2.val;</span><br><span class="line"> l2 = l2.next;</span><br><span class="line"> }</span><br><span class="line"> entered = (temp - temp % <span class="number">10</span>) / <span class="number">10</span>;</span><br><span class="line"> tail.val = temp % <span class="number">10</span>;</span><br><span class="line"> <span class="comment">// 循环内部的控制是为了排除最后的空节点</span></span><br><span class="line"> <span class="keyword">if</span> (l1 != <span class="literal">null</span> || l2 != <span class="literal">null</span> || entered != <span class="number">0</span>) {</span><br><span class="line"> tail.next = <span class="keyword">new</span> <span class="title class_">ListNode</span>();</span><br><span class="line"> tail = tail.next;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"><span class="comment">// tail = null;</span></span><br><span class="line"> <span class="keyword">return</span> root;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里唯二需要注意的就是两个点，一个是循环条件需要包含进位值还存在的情况，还有一个是最后一个节点，如果是空的了，就不要在 new 一个出来了，写的比较挫</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
<category>Java</category>
|
|
<category>linked list</category>
|
|
<category>java</category>
|
|
<category>linked list</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
<tag>linked list</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 48 旋转图像(Rotate Image) 题解分析</title>
|
|
<url>/2021/05/01/Leetcode-48-%E6%97%8B%E8%BD%AC%E5%9B%BE%E5%83%8F-Rotate-Image-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h3 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h3><p>You are given an n x n 2D <code>matrix</code> representing an image, rotate the image by 90 degrees (clockwise).</p>
|
|
<p>You have to rotate the image in-place, which means you have to modify the input 2D matrix directly. <strong>DO NOT</strong> allocate another 2D matrix and do the rotation.<br><img data-src="https://img.nicksxs.me/uPic/p8lf4Y.png"><br>如图，这道题以前做过，其实一看有点蒙，好像规则很容易描述，但是代码很难写，因为要类似于贪吃蛇那样，后来想着应该会有一些特殊的技巧，比如翻转等</p>
|
|
<h3 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h3><p>直接上码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">rotate</span><span class="params">(<span class="type">int</span>[][] matrix)</span> {</span><br><span class="line"> <span class="comment">// 这里真的傻了，长宽应该是一致的，所以取一次就够了</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">lengthX</span> <span class="operator">=</span> matrix[<span class="number">0</span>].length;</span><br><span class="line"> <span class="type">int</span> <span class="variable">lengthY</span> <span class="operator">=</span> matrix.length;</span><br><span class="line"> <span class="type">int</span> temp;</span><br><span class="line"> System.out.println(lengthY - (lengthY % <span class="number">2</span>) / <span class="number">2</span>);</span><br><span class="line"> <span class="comment">// 这里除错了，应该是减掉余数再除 2</span></span><br><span class="line"><span class="comment">// for (int i = 0; i < lengthY - (lengthY % 2) / 2; i++) {</span></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 1 2 3 7 8 9</span></span><br><span class="line"><span class="comment"> * 4 5 6 => 4 5 6 先沿着 4 5 6 上下交换</span></span><br><span class="line"><span class="comment"> * 7 8 9 1 2 3</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < (lengthY - (lengthY % <span class="number">2</span>)) / <span class="number">2</span>; i++) {</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">j</span> <span class="operator">=</span> <span class="number">0</span>; j < lengthX; j++) {</span><br><span class="line"> temp = matrix[i][j];</span><br><span class="line"> matrix[i][j] = matrix[lengthY-i-<span class="number">1</span>][j];</span><br><span class="line"> matrix[lengthY-i-<span class="number">1</span>][j] = temp;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 7 8 9 7 4 1</span></span><br><span class="line"><span class="comment"> * 4 5 6 => 8 5 2 这里再沿着 7 5 3 这条对角线交换</span></span><br><span class="line"><span class="comment"> * 1 2 3 9 6 3</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < lengthX; i++) {</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">j</span> <span class="operator">=</span> <span class="number">0</span>; j <= i; j++) {</span><br><span class="line"> <span class="keyword">if</span> (i == j) {</span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"> temp = matrix[i][j];</span><br><span class="line"> matrix[i][j] = matrix[j][i];</span><br><span class="line"> matrix[j][i] = temp;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>还没到可以直接归纳题目类型的水平，主要是几年前做过，可能有那么点模糊的记忆，当然应该也有直接转的方法</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
<category>Rotate Image</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
<tag>Rotate Image</tag>
|
|
<tag>矩阵</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 698 划分为k个相等的子集 ( Partition to K Equal Sum Subsets *Medium* ) 题解分析</title>
|
|
<url>/2022/06/19/Leetcode-698-%E5%88%92%E5%88%86%E4%B8%BAk%E4%B8%AA%E7%9B%B8%E7%AD%89%E7%9A%84%E5%AD%90%E9%9B%86-Partition-to-K-Equal-Sum-Subsets-Medium-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h3 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h3><p>Given an integer array <code>nums</code> and an integer <code>k</code>, return <code>true</code> if it is possible to divide this array into <code>k</code> non-empty subsets whose sums are all equal.</p>
|
|
<h4 id="示例"><a href="#示例" class="headerlink" title="示例"></a>示例</h4><p><strong>Example 1:</strong> </p>
|
|
<blockquote>
|
|
<p><strong>Input:</strong> nums = [4,3,2,3,5,2,1], k = 4<br><strong>Output:</strong> true<br><strong>Explanation:</strong> It is possible to divide it into 4 subsets (5), (1, 4), (2,3), (2,3) with equal sums. </p>
|
|
</blockquote>
|
|
<p><strong>Example 2:</strong></p>
|
|
<blockquote>
|
|
<p><strong>Input:</strong> nums = [1,2,3,4], k = 3<br><strong>Output:</strong> false</p>
|
|
</blockquote>
|
|
<p><strong>Constraints:</strong></p>
|
|
<ul>
|
|
<li>1 <= k <= nums.length <= 16</li>
|
|
<li>1 <= nums[i] <= 10^4</li>
|
|
<li>The frequency of each element is in the range [1, 4].</li>
|
|
</ul>
|
|
<h3 id="解析"><a href="#解析" class="headerlink" title="解析"></a>解析</h3><p>看到这个题一开始以为挺简单，但是仔细想想问题还是挺多的，首先是分成 k 组，但是数量不限，应该需要用到回溯的方式，同时对于时间和空间复杂度也有要求，一开始这个代码是超时的，我也试了下 leetcode 上 discussion 里 vote 最高的提交也是超时的，不过看 discussion 里的帖子，貌似是后面加了一些条件，可以帮忙提高执行效率，第三条提示不太清楚意图，具体可以看下代码</p>
|
|
<h3 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">canPartitionKSubsets</span><span class="params">(<span class="type">int</span>[] nums, <span class="type">int</span> k)</span> {</span><br><span class="line"> <span class="keyword">if</span> (k == <span class="number">1</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">int</span> <span class="variable">sum</span> <span class="operator">=</span> <span class="number">0</span>, n;</span><br><span class="line"> n = nums.length;</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> num : nums) {</span><br><span class="line"> sum += num;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (sum % k != <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">int</span> <span class="variable">avg</span> <span class="operator">=</span> sum / k;</span><br><span class="line"> <span class="comment">// 排序</span></span><br><span class="line"> Arrays.sort(nums);</span><br><span class="line"> <span class="comment">// 做个前置判断，如果最大值超过分组平均值了就可以返回 false 了</span></span><br><span class="line"> <span class="keyword">if</span> (nums[n - <span class="number">1</span>] > avg) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 这里取了个巧，先将数组中元素就等于分组平均值的直接排除了</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">calculated</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> n - <span class="number">1</span>; i > <span class="number">0</span>; i--) {</span><br><span class="line"> <span class="keyword">if</span> (nums[i] == avg) {</span><br><span class="line"> k--;</span><br><span class="line"> calculated++;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">int</span>[] bucket = <span class="keyword">new</span> <span class="title class_">int</span>[k];</span><br><span class="line"> <span class="comment">// 初始化 bucket</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < k; i++) {</span><br><span class="line"> bucket[i] = avg;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 提前做下边界判断</span></span><br><span class="line"> <span class="keyword">if</span> (nums[n - <span class="number">1</span>] > avg) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> backTraversal(nums, bucket, k, n - <span class="number">1</span> - calculated);</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">private</span> <span class="type">boolean</span> <span class="title function_">backTraversal</span><span class="params">(<span class="type">int</span>[] nums, <span class="type">int</span>[] bucket, <span class="type">int</span> k, <span class="type">int</span> cur)</span> {</span><br><span class="line"> <span class="keyword">if</span> (cur < <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < k; i++) {</span><br><span class="line"> <span class="keyword">if</span> (bucket[i] == nums[cur] || bucket[i] >= nums[cur] + nums[<span class="number">0</span>]) {</span><br><span class="line"> <span class="comment">// 判断如果当前 bucket[i] 剩余的数字等于nums[cur], 即当前bucket已经满了</span></span><br><span class="line"> <span class="comment">// 或者如果当前 bucket[i] 剩余的数字大于等于 nums[cur] + nums[0] ，</span></span><br><span class="line"> <span class="comment">// 因为nums 在经过排序后 nums[0]是最小值，如果加上 nums[0] 都已经超过bucket[i] 了，</span></span><br><span class="line"> <span class="comment">// 那当前bucket[i] 肯定是没法由包含 nums[cur] 的组合组成一个满足和为前面 s/k 的组合了</span></span><br><span class="line"> <span class="comment">// 这里判断的是 nums[cur] ,如果第一次 k 次循环都不符合其实就返回 false 了</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">// 而如果符合，就将 bucket[i] 减去 nums[cur] 再次进入递归，</span></span><br><span class="line"> <span class="comment">// 这里进入递归有个收敛参数就是 cur - 1，因为其实判断 cur 递减作为一个结束条件</span></span><br><span class="line"> bucket[i] -= nums[cur];</span><br><span class="line"> <span class="comment">// 符合条件，这里对应着入口，当 cur 被减到 0 了，就表示都符合了因为是根据所有值的和 s 和 k 组除出来的平均值，当所有数都通过前面的 if 判断符合了，并且每个数字都使用了，</span></span><br><span class="line"> <span class="comment">// 即说明已经符合要求了</span></span><br><span class="line"> <span class="keyword">if</span> (backTraversal(nums, bucket, k, cur - <span class="number">1</span>)) <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> <span class="comment">// 这边是个回退机制，如果前面 nums[cur]没办法组合成和为平均值的话就减掉进入下一个循环</span></span><br><span class="line"> bucket[i] += nums[cur];</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>最后贴个图<br><img data-src="https://img.nicksxs.me/uPic/pjOLSM.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 747 至少是其他数字两倍的最大数 ( Largest Number At Least Twice of Others *Easy* ) 题解分析</title>
|
|
<url>/2022/10/02/Leetcode-747-%E8%87%B3%E5%B0%91%E6%98%AF%E5%85%B6%E4%BB%96%E6%95%B0%E5%AD%97%E4%B8%A4%E5%80%8D%E7%9A%84%E6%9C%80%E5%A4%A7%E6%95%B0-Largest-Number-At-Least-Twice-of-Others-Easy-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h3 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h3><p>You are given an integer array <code>nums</code> where the largest integer is <strong>unique</strong>.</p>
|
|
<p>Determine whether the largest element in the array is <strong>at least twice</strong> as much as every other number in the array. If it is, return the <code>index</code> of the largest element, or return <code>-1</code> otherwise.<br>确认在数组中的最大数是否是其余任意数的两倍大及以上，如果是返回索引，如果不是返回-1</p>
|
|
<h3 id="示例"><a href="#示例" class="headerlink" title="示例"></a>示例</h3><h4 id="Example-1"><a href="#Example-1" class="headerlink" title="Example 1:"></a>Example 1:</h4><blockquote>
|
|
<p><strong>Input:</strong> nums = [3,6,1,0]<br><strong>Output:</strong> 1<br><strong>Explanation:</strong> 6 is the largest integer.<br>For every other number in the array x, 6 is at least twice as big as x.<br>The index of value 6 is 1, so we return 1. </p>
|
|
</blockquote>
|
|
<h3 id="Example-2"><a href="#Example-2" class="headerlink" title="Example 2:"></a>Example 2:</h3><blockquote>
|
|
<p><strong>Input:</strong> nums = [1,2,3,4]<br><strong>Output:</strong> -1<br><strong>Explanation:</strong> 4 is less than twice the value of 3, so we return -1. </p>
|
|
</blockquote>
|
|
<h3 id="提示"><a href="#提示" class="headerlink" title="提示:"></a>提示:</h3><ul>
|
|
<li><code>2 <= nums.length <= 50</code></li>
|
|
<li><code>0 <= nums[i] <= 100</code></li>
|
|
<li>The largest element in <code>nums</code> is unique.</li>
|
|
</ul>
|
|
<h3 id="简要解析"><a href="#简要解析" class="headerlink" title="简要解析"></a>简要解析</h3><p>这个题easy是题意也比较简单，找最大值，并且最大值是其他任意值的两倍及以上，其实就是找最大值跟次大值，比较下就好了</p>
|
|
<h3 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">int</span> <span class="title function_">dominantIndex</span><span class="params">(<span class="type">int</span>[] nums)</span> {</span><br><span class="line"> <span class="type">int</span> <span class="variable">largest</span> <span class="operator">=</span> Integer.MIN_VALUE;</span><br><span class="line"> <span class="type">int</span> <span class="variable">second</span> <span class="operator">=</span> Integer.MIN_VALUE;</span><br><span class="line"> <span class="type">int</span> <span class="variable">largestIndex</span> <span class="operator">=</span> -<span class="number">1</span>;</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < nums.length; i++) {</span><br><span class="line"> <span class="comment">// 如果有最大的就更新，同时更新最大值和第二大的</span></span><br><span class="line"> <span class="keyword">if</span> (nums[i] > largest) {</span><br><span class="line"> second = largest;</span><br><span class="line"> largest = nums[i];</span><br><span class="line"> largestIndex = i;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (nums[i] > second) {</span><br><span class="line"> <span class="comment">// 没有超过最大的，但是比第二大的更大就更新第二大的</span></span><br><span class="line"> second = nums[i];</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 判断下是否符合题目要求，要是所有值的两倍及以上</span></span><br><span class="line"> <span class="keyword">if</span> (largest >= <span class="number">2</span> * second) {</span><br><span class="line"> <span class="keyword">return</span> largestIndex;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> -<span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<h3 id="通过图"><a href="#通过图" class="headerlink" title="通过图"></a>通过图</h3><p>第一次错了是把第二大的情况只考虑第一种，也有可能最大值完全没经过替换就变成最大值了<br><img data-src="https://img.nicksxs.me/uPic/WechatIMG1065.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 83 删除排序链表中的重复元素 ( Remove Duplicates from Sorted List *Easy* ) 题解分析</title>
|
|
<url>/2022/03/13/Leetcode-83-%E5%88%A0%E9%99%A4%E6%8E%92%E5%BA%8F%E9%93%BE%E8%A1%A8%E4%B8%AD%E7%9A%84%E9%87%8D%E5%A4%8D%E5%85%83%E7%B4%A0-Remove-Duplicates-from-Sorted-List-Easy-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h3 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h3><p>给定一个已排序的链表的头 <code>head</code> ， 删除所有重复的元素，使每个元素只出现一次 。返回 已排序的链表 。<br> PS：注意已排序，还有返回也要已排序</p>
|
|
<h4 id="示例-1："><a href="#示例-1：" class="headerlink" title="示例 1："></a>示例 1：</h4><p><img data-src="https://img.nicksxs.me/uPic/gOhFFx.jpg"></p>
|
|
<blockquote>
|
|
<p>输入：head = [1,1,2]<br>输出：[1,2]</p>
|
|
</blockquote>
|
|
<h4 id="示例-2："><a href="#示例-2：" class="headerlink" title="示例 2："></a>示例 2：</h4><p><img data-src="https://img.nicksxs.me/uPic/Jy3J8a.jpg"></p>
|
|
<blockquote>
|
|
<p>输入：head = [1,1,2,3,3]<br>输出：[1,2,3]</p>
|
|
</blockquote>
|
|
<h3 id="提示："><a href="#提示：" class="headerlink" title="提示："></a>提示：</h3><ul>
|
|
<li>链表中节点数目在范围 <code>[0, 300]</code> 内</li>
|
|
<li><code>-100 <= Node.val <= 100</code></li>
|
|
<li>题目数据保证链表已经按 <strong>升序</strong> 排列</li>
|
|
</ul>
|
|
<h3 id="分析与题解"><a href="#分析与题解" class="headerlink" title="分析与题解"></a>分析与题解</h3><p>这题其实是比较正常的 easy 级别的题目，链表已经排好序了，如果还带一个排序就更复杂一点，<br>只需要前后项做个对比，如果一致则移除后项，因为可能存在多个重复项，所以只有在前后项不同<br>时才会更新被比较项</p>
|
|
<h4 id="code"><a href="#code" class="headerlink" title="code"></a>code</h4><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> ListNode <span class="title function_">deleteDuplicates</span><span class="params">(ListNode head)</span> {</span><br><span class="line"> <span class="comment">// 链表头是空的或者只有一个头结点，就不用处理了</span></span><br><span class="line"> <span class="keyword">if</span> (head == <span class="literal">null</span> || head.next == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> head;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">ListNode</span> <span class="variable">tail</span> <span class="operator">=</span> head;</span><br><span class="line"> <span class="comment">// 以处理节点还有后续节点作为循环边界条件</span></span><br><span class="line"> <span class="keyword">while</span> (tail.next != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">ListNode</span> <span class="variable">temp</span> <span class="operator">=</span> tail.next;</span><br><span class="line"> <span class="comment">// 如果前后相同，那么可以跳过这个节点，将 Tail ----> temp ---> temp.next </span></span><br><span class="line"> <span class="comment">// 更新成 Tail ----> temp.next</span></span><br><span class="line"> <span class="keyword">if</span> (temp.val == tail.val) {</span><br><span class="line"> tail.next = temp.next;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// 不相同，则更新 tail</span></span><br><span class="line"> tail = tail.next;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 最后返回头结点</span></span><br><span class="line"> <span class="keyword">return</span> head;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>链表应该是个需要反复的训练的数据结构，因为涉及到前后指针，然后更新操作，判空等，<br>我在这块也是掌握的不太好，需要多练习。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
<tag>Remove Duplicates from Sorted List</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>leetcode no.3</title>
|
|
<url>/2015/04/15/Leetcode-No-3/</url>
|
|
<content><![CDATA[<p>**Longest Substring Without Repeating Characters **</p>
|
|
<span id="more"></span>
|
|
<h3 id="description"><a href="#description" class="headerlink" title="description"></a>description</h3><p>Given a string, find the length of the longest substring without repeating characters.<br>For example, the longest substring without repeating letters for “abcabcbb” is “abc”,<br>which the length is 3. For “bbbbb” the longest substring is “b”, with the length of 1. </p>
|
|
<h3 id="分析"><a href="#分析" class="headerlink" title="分析"></a>分析</h3><p><a href="http://www.cnblogs.com/dollarzhaole/p/3155712.html">源码</a>这次是参考了这个代码，<br>tail 表示的当前子串的起始点位置，tail从-1开始就包括的串的长度是1的边界。其实我<br>也是猜的（逃</p>
|
|
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="type">int</span> ct[<span class="number">256</span>];</span><br><span class="line"> <span class="built_in">memset</span>(ct, <span class="number">-1</span>, <span class="built_in">sizeof</span>(ct));</span><br><span class="line"> <span class="type">int</span> tail = <span class="number">-1</span>;</span><br><span class="line"> <span class="type">int</span> max = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> i = <span class="number">0</span>; i < s.<span class="built_in">size</span>(); i++){</span><br><span class="line"> <span class="keyword">if</span> (ct[s[i]] > tail)</span><br><span class="line"> tail = ct[s[i]];</span><br><span class="line"> <span class="keyword">if</span> (i - tail > max)</span><br><span class="line"> max = i - tail;</span><br><span class="line"> ct[s[i]] = i;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> max;</span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>c++</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Linux 下 grep 命令的一点小技巧</title>
|
|
<url>/2020/08/06/Linux-%E4%B8%8B-grep-%E5%91%BD%E4%BB%A4%E7%9A%84%E4%B8%80%E7%82%B9%E5%B0%8F%E6%8A%80%E5%B7%A7/</url>
|
|
<content><![CDATA[<p>用了比较久的 grep 命令，其实都只是用了最最基本的功能来查日志，</p>
|
|
<p>譬如</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"></span><br><span class="line">grep 'xxx' xxxx.log</span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后有挺多情况比如想要找日志里带一些符号什么的，就需要用到一些特殊的</p>
|
|
<p>比如这样<code>\"userId\":\"123456\"</code>，因为比如用户 ID 有时候会跟其他的 id 一样，只用具体的值 123456 来查的话干扰信息太多了，如果直接这样</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"></span><br><span class="line">grep '\"userId\":\"123456\"' xxxx.log</span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
|
|
<p>好像不行，盲猜就是符号的问题，特别是<code>\</code>和<code>"</code>这两个，</p>
|
|
<p>之前一直是想试一下，但是没成功，昨天在排查一个问题的时候发现了，只要把这些都转义了就行了</p>
|
|
<p><code>grep '\\\"userId\\\":\\\"123456\\\"' xxxx.log</code></p>
|
|
<p><img data-src="https://img.nicksxs.me/uPic/sUdv2K.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Linux</category>
|
|
<category>命令</category>
|
|
<category>小技巧</category>
|
|
<category>grep</category>
|
|
<category>grep</category>
|
|
<category>查日志</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>linux</tag>
|
|
<tag>grep</tag>
|
|
<tag>转义</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 885 螺旋矩阵 III ( Spiral Matrix III *Medium* ) 题解分析</title>
|
|
<url>/2022/08/23/Leetcode-885-%E8%9E%BA%E6%97%8B%E7%9F%A9%E9%98%B5-III-Spiral-Matrix-III-Medium-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h3 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h3><p>You start at the cell (rStart, cStart) of an rows x cols grid facing east. The northwest corner is at the first row and column in the grid, and the southeast corner is at the last row and column.</p>
|
|
<p>You will walk in a clockwise spiral shape to visit every position in this grid. Whenever you move outside the grid’s boundary, we continue our walk outside the grid (but may return to the grid boundary later.). Eventually, we reach all rows * cols spaces of the grid.</p>
|
|
<p>Return an array of coordinates representing the positions of the grid in the order you visited them.</p>
|
|
<h4 id="Example-1"><a href="#Example-1" class="headerlink" title="Example 1:"></a>Example 1:</h4><blockquote>
|
|
<p><strong>Input:</strong> rows = 1, cols = 4, rStart = 0, cStart = 0<br><strong>Output:</strong> [[0,0],[0,1],[0,2],[0,3]] </p>
|
|
</blockquote>
|
|
<h4 id="Example-2"><a href="#Example-2" class="headerlink" title="Example 2:"></a>Example 2:</h4><blockquote>
|
|
<p><strong>Input:</strong> rows = 5, cols = 6, rStart = 1, cStart = 4<br><strong>Output:</strong> [[1,4],[1,5],[2,5],[2,4],[2,3],[1,3],[0,3],[0,4],[0,5],[3,5],[3,4],[3,3],[3,2],[2,2],[1,2],[0,2],[4,5],[4,4],[4,3],[4,2],[4,1],[3,1],[2,1],[1,1],[0,1],[4,0],[3,0],[2,0],[1,0],[0,0]] </p>
|
|
</blockquote>
|
|
<p><strong>Constraints:</strong> </p>
|
|
<ul>
|
|
<li><code>1 <= rows, cols <= 100</code> </li>
|
|
<li><code>0 <= rStart < rows</code> </li>
|
|
<li><code>0 <= cStart < cols</code></li>
|
|
</ul>
|
|
<h3 id="简析"><a href="#简析" class="headerlink" title="简析"></a>简析</h3><p>这个题主要是要相同螺旋矩阵的转变方向的边界判断，已经相同步长会行进两次这个规律，写代码倒不复杂</p>
|
|
<h3 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">int</span>[][] spiralMatrixIII(<span class="type">int</span> rows, <span class="type">int</span> cols, <span class="type">int</span> rStart, <span class="type">int</span> cStart) {</span><br><span class="line"> <span class="type">int</span> <span class="variable">size</span> <span class="operator">=</span> rows * cols;</span><br><span class="line"> <span class="type">int</span> <span class="variable">x</span> <span class="operator">=</span> rStart, y = cStart;</span><br><span class="line"> <span class="comment">// 返回的二维矩阵</span></span><br><span class="line"> <span class="type">int</span>[][] matrix = <span class="keyword">new</span> <span class="title class_">int</span>[size][<span class="number">2</span>];</span><br><span class="line"> <span class="comment">// 传入的参数就是入口第一个</span></span><br><span class="line"> matrix[<span class="number">0</span>][<span class="number">0</span>] = rStart;</span><br><span class="line"> matrix[<span class="number">0</span>][<span class="number">1</span>] = cStart;</span><br><span class="line"> <span class="comment">// 作为数量</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">z</span> <span class="operator">=</span> <span class="number">1</span>;</span><br><span class="line"> <span class="comment">// 步进，1，1，2，2，3，3，4 ... 螺旋矩阵的增长</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">a</span> <span class="operator">=</span> <span class="number">1</span>;</span><br><span class="line"> <span class="comment">// 方向 1 表示右，2 表示下，3 表示左，4 表示上</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">dir</span> <span class="operator">=</span> <span class="number">1</span>;</span><br><span class="line"> <span class="keyword">while</span> (z < size) {</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < <span class="number">2</span>; i++) {</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> j= <span class="number">0</span>; j < a; j++) {</span><br><span class="line"> <span class="comment">// 处理方向</span></span><br><span class="line"> <span class="keyword">if</span> (dir % <span class="number">4</span> == <span class="number">1</span>) {</span><br><span class="line"> y++;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (dir % <span class="number">4</span> == <span class="number">2</span>) {</span><br><span class="line"> x++;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (dir % <span class="number">4</span> == <span class="number">3</span>) {</span><br><span class="line"> y--;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> x--;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 如果在实际矩阵内</span></span><br><span class="line"> <span class="keyword">if</span> (x < rows && y < cols && x >= <span class="number">0</span> && y >= <span class="number">0</span>) {</span><br><span class="line"> matrix[z][<span class="number">0</span>] = x;</span><br><span class="line"> matrix[z][<span class="number">1</span>] = y;</span><br><span class="line"> z++;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 转变方向</span></span><br><span class="line"> dir++;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 步进++</span></span><br><span class="line"> a++;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> matrix;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<h3 id="结果"><a href="#结果" class="headerlink" title="结果"></a>结果</h3><p><img data-src="https://img.nicksxs.me/uPic/2w6vbW.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>MFC 模态对话框</title>
|
|
<url>/2014/12/24/MFC%20%E6%A8%A1%E6%80%81%E5%AF%B9%E8%AF%9D%E6%A1%86/</url>
|
|
<content><![CDATA[<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="type">void</span> <span class="title">CTestDialog::OnBnClickedOk</span><span class="params">()</span> </span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> CString m_SrcTest;</span><br><span class="line"> <span class="type">int</span> nIndex = m_CbTest.<span class="built_in">GetCurSel</span>(); </span><br><span class="line"> m_CbTest.<span class="built_in">GetLBText</span>(nIndex, m_SrcTest); </span><br><span class="line"> <span class="built_in">OnOK</span>();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>模态对话框弹出确定后，在弹出对话框时新建的类及其变量会存在，但是对于其中的控件<br>对象无法调用函数，即如果要在主对话框中获得弹出对话框的Combo box选中值的话，需<br>要在弹出 对话框的确定函数内将其值取出，赋值给弹出对话框的公有变量，这样就可以<br>在主对话框类得到值。 </p>
|
|
]]></content>
|
|
<categories>
|
|
<category>C++</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>c++</tag>
|
|
<tag>mfc</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Number of 1 Bits</title>
|
|
<url>/2015/03/11/Number-Of-1-Bits/</url>
|
|
<content><![CDATA[<h3 id="Number-of-1-Bits"><a href="#Number-of-1-Bits" class="headerlink" title="Number of 1 Bits "></a><a href="https://leetcode.com/problems/number-of-1-bits/">Number of 1 Bits </a></h3><h4 id="Write-a-function-that-takes-an-unsigned-integer-and-returns-the-number-of-’1’-bits-it-has-also-known-as-the-Hamming-weight-For-example-the-32-bit-integer-‘11’-has-binary-representation-00000000000000000000000000001011-so-the-function-should-return-3"><a href="#Write-a-function-that-takes-an-unsigned-integer-and-returns-the-number-of-’1’-bits-it-has-also-known-as-the-Hamming-weight-For-example-the-32-bit-integer-‘11’-has-binary-representation-00000000000000000000000000001011-so-the-function-should-return-3" class="headerlink" title="Write a function that takes an unsigned integer and returns the number of ’1’ bits it has (also known as the Hamming weight). For example, the 32-bit integer ‘11’ has binary representation 00000000000000000000000000001011, so the function should return 3."></a>Write a function that takes an unsigned integer and returns the number of ’1’ bits it has (also known as the Hamming weight). For example, the 32-bit integer ‘11’ has binary representation <code>00000000000000000000000000001011</code>, so the function should return 3.</h4><span id="more"></span>
|
|
|
|
<h3 id="分析"><a href="#分析" class="headerlink" title="分析"></a>分析</h3><p>从1位到2位到4位逐步的交换</p>
|
|
<hr>
|
|
<h3 id="code"><a href="#code" class="headerlink" title="code"></a>code</h3><figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="type">int</span> <span class="title">hammingWeight</span><span class="params">(<span class="type">uint32_t</span> n)</span> </span>{</span><br><span class="line"> <span class="type">const</span> <span class="type">uint32_t</span> m1 = <span class="number">0x55555555</span>; <span class="comment">//binary: 0101... </span></span><br><span class="line"> <span class="type">const</span> <span class="type">uint32_t</span> m2 = <span class="number">0x33333333</span>; <span class="comment">//binary: 00110011.. </span></span><br><span class="line"> <span class="type">const</span> <span class="type">uint32_t</span> m4 = <span class="number">0x0f0f0f0f</span>; <span class="comment">//binary: 4 zeros, 4 ones ... </span></span><br><span class="line"> <span class="type">const</span> <span class="type">uint32_t</span> m8 = <span class="number">0x00ff00ff</span>; <span class="comment">//binary: 8 zeros, 8 ones ... </span></span><br><span class="line"> <span class="type">const</span> <span class="type">uint32_t</span> m16 = <span class="number">0x0000ffff</span>; <span class="comment">//binary: 16 zeros, 16 ones ... </span></span><br><span class="line"> </span><br><span class="line"> n = (n & m1 ) + ((n >> <span class="number">1</span>) & m1 ); <span class="comment">//put count of each 2 bits into those 2 bits </span></span><br><span class="line"> n = (n & m2 ) + ((n >> <span class="number">2</span>) & m2 ); <span class="comment">//put count of each 4 bits into those 4 bits </span></span><br><span class="line"> n = (n & m4 ) + ((n >> <span class="number">4</span>) & m4 ); <span class="comment">//put count of each 8 bits into those 8 bits </span></span><br><span class="line"> n = (n & m8 ) + ((n >> <span class="number">8</span>) & m8 ); <span class="comment">//put count of each 16 bits into those 16 bits </span></span><br><span class="line"> n = (n & m16) + ((n >> <span class="number">16</span>) & m16); <span class="comment">//put count of each 32 bits into those 32 bits </span></span><br><span class="line"> <span class="keyword">return</span> n; </span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>c++</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Maven实用小技巧</title>
|
|
<url>/2020/02/16/Maven%E5%AE%9E%E7%94%A8%E5%B0%8F%E6%8A%80%E5%B7%A7/</url>
|
|
<content><![CDATA[<p>Maven 翻译为”专家”、”内行”，是 Apache 下的一个纯 Java 开发的开源项目。基于项目对象模型（缩写：POM）概念，Maven利用一个中央信息片断能管理一个项目的构建、报告和文档等步骤。</p>
|
|
<p>Maven 是一个项目管理工具，可以对 Java 项目进行构建、依赖管理。</p>
|
|
<p>Maven 也可被用于构建和管理各种项目，例如 C#，Ruby，Scala 和其他语言编写的项目。Maven 曾是 Jakarta 项目的子项目，现为由 Apache 软件基金会主持的独立 Apache 项目。</p>
|
|
<p>maven也是我们日常项目中实用的包管理工具，相比以前需要用把包下载下来，放进 lib 中，在平时工作中使用的话，其实像 idea 这样的 ide 工具都会自带 maven 工具和插件</p>
|
|
<h3 id="maven的基本操作"><a href="#maven的基本操作" class="headerlink" title="maven的基本操作"></a>maven的基本操作</h3><ul>
|
|
<li><code>mvn -v</code><br> 查看 maven 信息</li>
|
|
<li><code>mvn compile</code><br> 将 Java 编译成 class 文件</li>
|
|
<li><code>mvn test</code><br> 执行 test 包下的测试用例</li>
|
|
<li><code>mvn package</code><br> 将项目打成 jar 包</li>
|
|
<li><code>mvn clean</code><br> 删除package 在 target 目录下面打出来的 jar 包和 target 目录</li>
|
|
<li><code>mvn install</code><br> 将打出来的 jar 包复制到 maven 的本地仓库里</li>
|
|
<li><code>mvn deploy</code><br> 将打出来的 jar 包上传到远程仓库里</li>
|
|
</ul>
|
|
<h3 id="与-composer-对比"><a href="#与-composer-对比" class="headerlink" title="与 composer 对比"></a>与 composer 对比</h3><p>因为我也是个 PHP 程序员，所以对比一下两种语言，很容易想到在 PHP 的 composer 跟 Java 的 maven 是比较类似的作用，有一点两者是非常相似的，就是原仓库都是因为某些原因连接拉取都会很慢，所以像 composer 会有一些国内源，前阵子阿里也出了一个，类似的 maven 一般也会使用阿里的镜像仓库，通过在 setting.xml 文件中的设置</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line"><mirrors></span><br><span class="line"> <mirror></span><br><span class="line"> <id>aliyun</id></span><br><span class="line"> <name>aliyun maven</name></span><br><span class="line"> <url>http://maven.aliyun.com/nexus/content/groups/public/</url></span><br><span class="line"> <mirrorOf>central</mirrorOf></span><br><span class="line"> </mirror> </span><br><span class="line"></mirrors></span><br></pre></td></tr></table></figure>
|
|
<p>这算是个尴尬的共同点，然后因为 PHP 是解释型脚本语言，所以 php 打出来的 composer 包其实就是个 php 代码包，使用SPL Autoload等方式加载代码包，maven 包则是经过编译的 class 包，还有一点是 composer 也可以直接使用 github 地址作为包代码的拉取源，这点也是比较大的区别，maven使用 pom 文件管理依赖</p>
|
|
<h3 id="maven-的个人小技巧"><a href="#maven-的个人小技巧" class="headerlink" title="maven 的个人小技巧"></a>maven 的个人小技巧</h3><ul>
|
|
<li>maven 拉取依赖时，同时将 snapshot 也更新了，就是 <code>mvn compile</code>加个<code>-U</code>参数，如果还不行就需要将本地仓库的 snapshot 删掉，<br>这个命令的 help 命令解释是 -U,–update-snapshots Forces a check for missing releases and updated snapshots on<br>remote repositories，这个在日常使用中还是很经常使用的</li>
|
|
<li>maven 出现依赖冲突的时候的解决方法<br>首先是依赖分析，使用<code>mvn dependency:tree</code>分析下依赖关系，如果要找具体某个包的依赖引用关系可以使用<code>mvn dependency:tree -Dverbose -Dincludes=org.springframework:spring-webmvc</code>命令进行分析，如果发现有冲突的依赖关系，本身 maven 中依赖引用有相对的顺序，大致来说是引用路径短的优先，pom 文件中定义的顺序优先，如果要把冲突的包排除掉可以在 pom 中用<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line"><exclusions></span><br><span class="line"> <exclusion></span><br><span class="line"> <groupId>ch.qos.logback</groupId></span><br><span class="line"> <artifactId>logback-classic</artifactId></span><br><span class="line"> </exclusion></span><br><span class="line"></exclusions></span><br></pre></td></tr></table></figure>
|
|
将冲突的包排除掉</li>
|
|
<li>maven 依赖的 jdk 版本管理<br>前面介绍的<code>mvn -v</code>可以查看 maven 的安装信息<br>比如<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)</span><br><span class="line">Maven home: /usr/local/Cellar/maven/3.6.3_1/libexec</span><br><span class="line">Java version: 1.8.0_201, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/jdk1.8.0_201.jdk/Contents/Home/jre</span><br><span class="line">Default locale: zh_CN, platform encoding: UTF-8</span><br><span class="line">OS name: "mac os x", version: "10.14.6", arch: "x86_64", family: "mac"</span><br></pre></td></tr></table></figure>
|
|
这里可以看到用了 mac 自带的 jdk1.8，结合我之前碰到的一个问题，因为使用 homebrew 升级了 gradle，而 gradle 又依赖了 jdk13，因为这个 mvn 的 Java version 也变成 jdk13 了，然后 mvn 编译的时候出现了 <code>java.lang.ExceptionInInitializerError: com.sun.tools.javac.code.TypeTags</code>这个问题，所以需要把这个版本给改回来，但是咋改呢，网上搜来的一大堆都是在 pom 文件里的<br>source和 target 版本<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line"><build></span><br><span class="line"> <plugins></span><br><span class="line"><plugin></span><br><span class="line"> <groupId>org.apache.maven.plugins</groupId></span><br><span class="line"> <artifactId>maven-compiler-plugin</artifactId></span><br><span class="line"> <configuration></span><br><span class="line"> <source>1.8</source></span><br><span class="line"> <target>1.8</target></span><br><span class="line"> <encoding>UTF-8</encoding></span><br><span class="line"> </configuration></span><br><span class="line"></plugin></span><br><span class="line"> </plugins></span><br><span class="line"><build></span><br></pre></td></tr></table></figure>
|
|
或者修改 maven 的 setting.xml中的<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line"><profiles></span><br><span class="line"> <profile></span><br><span class="line"> <id>ngmm-nexus</id></span><br><span class="line"> <activation></span><br><span class="line"> <jdk>1.8</jdk></span><br><span class="line"> </activation></span><br><span class="line"> <properties></span><br><span class="line"> <maven.compiler.source>1.8</maven.compiler.source></span><br><span class="line"> <maven.compiler.target>1.8</maven.compiler.target></span><br><span class="line"> <maven.compiler.compilerVersion>1.8</maven.compiler.compilerVersion></span><br><span class="line"> </properties></span><br><span class="line"> </profile></span><br><span class="line"></profiles></span><br></pre></td></tr></table></figure>
|
|
但是这些都没啥用啊，真正有办法的是建个 <code>.mavenrc</code>，这个顾名思义就是 maven 的资源文件，类似于 <code>.bashrc</code>和<code>.zshrc</code>，在里面添加 MAVEN_HOME 和 JAVA_HOME,然后执行 <code>source .mavenrc</code>就 OK 啦</li>
|
|
</ul>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Maven</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Maven</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Path Sum</title>
|
|
<url>/2015/01/04/Path-Sum/</url>
|
|
<content><![CDATA[<h3 id="problem"><a href="#problem" class="headerlink" title="problem"></a>problem</h3><p>Given a binary tree and a sum, determine if the tree has a root-to-leaf path such that adding up all the values along the path equals the given sum.</p>
|
|
<span id="more"></span>
|
|
<p>For example:<br>Given the below binary tree and sum = 22,</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line"> 5</span><br><span class="line"> / \</span><br><span class="line"> 4 8</span><br><span class="line"> / / \</span><br><span class="line"> 11 13 4</span><br><span class="line"> / \ \</span><br><span class="line">7 2 1</span><br></pre></td></tr></table></figure>
|
|
<p>return true, as there exist a root-to-leaf path 5->4->11->2 which sum is 22.</p>
|
|
<h3 id="Analysis"><a href="#Analysis" class="headerlink" title="Analysis"></a>Analysis</h3><p>using simple deep first search</p>
|
|
<h3 id="code"><a href="#code" class="headerlink" title="code"></a>code</h3><figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment"> Definition for binary tree</span></span><br><span class="line"><span class="comment"> struct TreeNode {</span></span><br><span class="line"><span class="comment"> int val;</span></span><br><span class="line"><span class="comment"> TreeNode *left;</span></span><br><span class="line"><span class="comment"> TreeNode *right;</span></span><br><span class="line"><span class="comment"> TreeNode(int x) : val(x), left(NULL), right(NULL)}</span></span><br><span class="line"><span class="comment"> };</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">class</span> <span class="title class_">Solution</span> {</span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line"> <span class="function"><span class="type">bool</span> <span class="title">deep_first_search</span><span class="params">(TreeNode *node, <span class="type">int</span> sum, <span class="type">int</span> curSum)</span></span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="keyword">if</span> (node == <span class="literal">NULL</span>)</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">if</span> (node->left == <span class="literal">NULL</span> && node->right == <span class="literal">NULL</span>)</span><br><span class="line"> <span class="keyword">return</span> curSum + node->val == sum;</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">deep_first_search</span>(node->left, sum, curSum + node->val) || <span class="built_in">deep_first_search</span>(node->right, sum, curSum + node->val);</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="function"><span class="type">bool</span> <span class="title">hasPathSum</span><span class="params">(TreeNode *root, <span class="type">int</span> sum)</span> </span>{</span><br><span class="line"> <span class="comment">// Start typing your C/C++ solution below</span></span><br><span class="line"> <span class="comment">// DO NOT write int main() function</span></span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">deep_first_search</span>(root, sum, <span class="number">0</span>);</span><br><span class="line"> }</span><br><span class="line">};</span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>c++</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Redis_分布式锁</title>
|
|
<url>/2019/12/10/Redis-Part-1/</url>
|
|
<content><![CDATA[<p>今天看了一下 <code>redis</code> 分布式锁 <code>redlock</code> 的实现，简单记录下，</p>
|
|
<h2 id="加锁"><a href="#加锁" class="headerlink" title="加锁"></a>加锁</h2><p>原先我对 <code>redis</code> 锁的概念就是加锁使用 <code>setnx</code>，解锁使用 <code>lua</code> 脚本，但是 <code>setnx</code> 具体是啥，<code>lua</code> 脚本是啥不是很清楚<br>首先简单思考下这个问题，首先为啥不是先 <code>get</code> 一下 <code>key</code> 存不存在，然后再 <code>set</code> 一个 <code>key value</code>，因为加锁这个操作我们是要保证两点，一个是不能中途被打断，也就是说要原子性，如果是先 <code>get</code> 一下 <code>key</code>，如果不存在再 <code>set</code> 值的话，那就不是原子操作了；第二个是可不可以直接 <code>set</code> 值呢，显然不行，锁要保证唯一性，有且只能有一个线程或者其他应用单位获得该锁，正好 <code>setnx</code> 给了我们这种原子命令</p>
|
|
<p>然后是 <code>setnx</code> 的键和值分别是啥，键比较容易想到是要锁住的资源，比如 <code>user_id</code>, 这里有个我自己之前比较容易陷进去的误区，但是这个误区后<br>面再说，这里其实是把<code>user_id</code> 作为要锁住的资源，在我获得锁的时候别的线程不允许操作，以此保证业务的正确性，不会被多个线程同时修改，确定了键，再来看看值是啥，其实原先我认为值是啥都没关系，我只要锁住了，光键就够我用了，但是考虑下多个线程的问题，如果我这个线程加了锁，然后我因为 <code>gc</code> 停顿等原因卡死了，这个时候<code>redis</code> 的锁或者说就是 <code>redis</code> 的缓存已经过期了，这时候另一个线程获得锁成功，然后我这个线程又活过来了，然后我就仍然认为我拿着锁，我去对数据进行修改或者释放锁，是不是就出现问题了，所以是不是我们还需要一个东西来区分这个锁是哪个线程加的，所以我们可以将值设置成为一个线程独有识别的值，至少在相对长的一段时间内不会重复。</p>
|
|
<p>上面其实还有两个问题，一个是当 <code>gc</code> 超时时，我这个线程如何知道我手里的锁已经过期了，一种方法是我在加好锁之后就维护一个超时时间，这里其实还有个问题，不过跟第二个问题相关，就一起说了，就是设置超时时间，有些对于不是锁的 <code>redis</code> 缓存操作可以是先设置好值，然后在设置过期时间，那么这就又有上面说到的不是原子性的问题，那么就需要在同一条指令里把超时时间也设置了，幸好 <code>redis</code> 提供了这种支持</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">SET resource_name my_random_value NX PX 30000</span><br></pre></td></tr></table></figure>
|
|
<p>这里借鉴一下解释下，<code>resource_name</code>就是 key，代表要锁住的东西，<code>my_random_value</code>就是识别我这个线程的，<code>NX</code>代表只有在不存在的时候才设置，然后<code>PX 30000</code>表示超时时间是 30秒自动过期</p>
|
|
<p>PS：记录下我原先有的一个误区，是不是要用 key 来区分加锁的线程，这样只有一个用处，就是自身线程可以识别是否是自己加的锁，但是最大的问题是别的线程不知道，其实这个用户的出发点是我在担心前面提过的一个问题，就是当 gc 停顿后，我要去判断当前的这个锁是否是我加的，还有就是当释放锁的时候，如果保证不会错误释放了其他线程加的锁，但是这样附带很多其他问题，最大的就是其他线程怎么知道能不能加这个锁。</p>
|
|
<h2 id="解锁"><a href="#解锁" class="headerlink" title="解锁"></a>解锁</h2><p>当线程在锁过期之前就处理完了业务逻辑，那就可以提前释放这个锁，那么提前释放要怎么操作，直接<code>del key</code>显然是不行的，因为这样就是我前面想用线程随机值加资源名作为锁的初衷，我不能去释放别的线程加的锁，那么我要怎么办呢，先 <code>get</code> 一下看是不是我的？那又变成非原子的操作了，幸好redis 也考虑到了这个问题，给了<code>lua</code> 脚本来操作这种</p>
|
|
<figure class="highlight lua"><table><tr><td class="code"><pre><span class="line"><span class="keyword">if</span> redis.call(<span class="string">"get"</span>,KEYS[<span class="number">1</span>]) == ARGV[<span class="number">1</span>] <span class="keyword">then</span></span><br><span class="line"> <span class="keyword">return</span> redis.call(<span class="string">"del"</span>,KEYS[<span class="number">1</span>])</span><br><span class="line"><span class="keyword">else</span></span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure>
|
|
<p>这里的<code>KEYS[1]</code>就是前面加锁的<code>resource_name</code>,<code>ARGV[1]</code>就是线程的随机值<code>my_random_value</code></p>
|
|
<h2 id="多节点"><a href="#多节点" class="headerlink" title="多节点"></a>多节点</h2><p>前面说的其实是单节点 <code>redis</code> 作为分布式锁的情况，那么当我们的 <code>redis</code> 有多节点的情况呢，如果多节点下处于加锁或者解锁或者锁有效情况下<br><code>redis</code> 的某个节点宕掉了怎么办，这里就有一些需要思考的地方，是否单独搞一个单节点的 <code>redis</code>作为分布式锁专用的，但是如果这个单节点的挂了呢，还有就是成本问题，所以我们需要一个多节点的分布式锁方案<br>这里就引出了开头说到的<code>redlock</code>，这个可是 <code>redis</code>的作者写的, 他的加锁过程是分以下几步去做这个事情</p>
|
|
<ul>
|
|
<li>获取当前时间（毫秒数）。</li>
|
|
<li>按顺序依次向N个Redis节点执行获取锁的操作。这个获取操作跟前面基于单Redis节点的获取锁的过程相同，包含随机字符串my_random_value，也包含过期时间(比如PX 30000，即锁的有效时间)。为了保证在某个Redis节点不可用的时候算法能够继续运行，这个获取锁的操作还有一个超时时间(time out)，它要远小于锁的有效时间（几十毫秒量级）。客户端在向某个Redis节点获取锁失败以后，应该立即尝试下一个Redis节点。这里的失败，应该包含任何类型的失败，比如该Redis节点不可用，或者该Redis节点上的锁已经被其它客户端持有（注：Redlock原文中这里只提到了Redis节点不可用的情况，但也应该包含其它的失败情况）。</li>
|
|
<li>计算整个获取锁的过程总共消耗了多长时间，计算方法是用当前时间减去第1步记录的时间。如果客户端从大多数Redis节点（>= N/2+1）成功获取到了锁，并且获取锁总共消耗的时间没有超过锁的有效时间(lock validity time)，那么这时客户端才认为最终获取锁成功；否则，认为最终获取锁失败。</li>
|
|
<li>如果最终获取锁成功了，那么这个锁的有效时间应该重新计算，它等于最初的锁的有效时间减去第3步计算出来的获取锁消耗的时间。</li>
|
|
<li>如果最终获取锁失败了（可能由于获取到锁的Redis节点个数少于N/2+1，或者整个获取锁的过程消耗的时间超过了锁的最初有效时间），那么客户端应该立即向所有Redis节点发起释放锁的操作（即前面介绍的Redis Lua脚本）。<br>释放锁的过程比较简单：客户端向所有Redis节点发起释放锁的操作，不管这些节点当时在获取锁的时候成功与否。这里为什么要向所有的节点发送释放锁的操作呢，这里是因为有部分的节点的失败原因可能是加锁时阻塞，加锁成功的结果没有及时返回，所以为了防止这种情况还是需要向所有发起这个释放锁的操作。<br>初步记录就先到这。</li>
|
|
</ul>
|
|
]]></content>
|
|
<categories>
|
|
<category>Redis</category>
|
|
<category>Distributed Lock</category>
|
|
<category>C</category>
|
|
<category>Redis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>C</tag>
|
|
<tag>Redis</tag>
|
|
<tag>Distributed Lock</tag>
|
|
<tag>分布式锁</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Reverse Bits</title>
|
|
<url>/2015/03/11/Reverse-Bits/</url>
|
|
<content><![CDATA[<h3 id="Reverse-Bits"><a href="#Reverse-Bits" class="headerlink" title="Reverse Bits "></a><a href="https://leetcode.com/problems/reverse-bits/">Reverse Bits </a></h3><h4 id="Reverse-bits-of-a-given-32-bits-unsigned-integer"><a href="#Reverse-bits-of-a-given-32-bits-unsigned-integer" class="headerlink" title="Reverse bits of a given 32 bits unsigned integer."></a>Reverse bits of a given 32 bits unsigned integer.</h4><p>For example, given input 43261596 (represented in binary as 00000010100101000001111010011100), return 964176192 (represented in binary as 00111001011110000010100101000000).</p>
|
|
<span id="more"></span>
|
|
<p>Follow up:<br>If this function is called many times, how would you optimize it?</p>
|
|
<hr>
|
|
<h3 id="code"><a href="#code" class="headerlink" title="code"></a>code</h3><figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">Solution</span> {</span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line"> <span class="function"><span class="type">uint32_t</span> <span class="title">reverseBits</span><span class="params">(<span class="type">uint32_t</span> n)</span> </span>{</span><br><span class="line"> n = ((n >> <span class="number">1</span>) & <span class="number">0x55555555</span>) | ((n & <span class="number">0x55555555</span>) << <span class="number">1</span>);</span><br><span class="line"> n = ((n >> <span class="number">2</span>) & <span class="number">0x33333333</span>) | ((n & <span class="number">0x33333333</span>) << <span class="number">2</span>);</span><br><span class="line"> n = ((n >> <span class="number">4</span>) & <span class="number">0x0f0f0f0f</span>) | ((n & <span class="number">0x0f0f0f0f</span>) << <span class="number">4</span>);</span><br><span class="line"> n = ((n >> <span class="number">8</span>) & <span class="number">0x00ff00ff</span>) | ((n & <span class="number">0x00ff00ff</span>) << <span class="number">8</span>);</span><br><span class="line"> n = ((n >> <span class="number">16</span>) & <span class="number">0x0000ffff</span>) | ((n & <span class="number">0x0000ffff</span>) << <span class="number">16</span>);</span><br><span class="line"> <span class="keyword">return</span> n;</span><br><span class="line"> }</span><br><span class="line">};</span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>c++</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Reverse Integer</title>
|
|
<url>/2015/03/13/Reverse-Integer/</url>
|
|
<content><![CDATA[<h3 id="Reverse-Integer"><a href="#Reverse-Integer" class="headerlink" title="Reverse Integer"></a><a href="https://leetcode.com/problems/reverse-integer/">Reverse Integer</a></h3><h4 id="Reverse-digits-of-an-integer"><a href="#Reverse-digits-of-an-integer" class="headerlink" title="Reverse digits of an integer."></a>Reverse digits of an integer.</h4><p>Example1: x = 123, return 321<br>Example2: x = -123, return -321</p>
|
|
<span id="more"></span>
|
|
<h4 id="spoilers"><a href="#spoilers" class="headerlink" title="spoilers"></a>spoilers</h4><p>Have you thought about this?<br>Here are some good questions to ask before coding. Bonus points for you if you have already thought through this!</p>
|
|
<p>If the integer’s last digit is 0, what should the output be? ie, cases such as 10, 100.</p>
|
|
<p>Did you notice that the reversed integer might overflow? Assume the input is a 32-bit integer, then the reverse of 1000000003 overflows. How should you handle such cases?</p>
|
|
<p>For the purpose of this problem, assume that your function returns 0 when the reversed integer overflows.</p>
|
|
<hr>
|
|
<h3 id="code"><a href="#code" class="headerlink" title="code"></a>code</h3><figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">Solution</span> {</span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line"> <span class="function"><span class="type">int</span> <span class="title">reverse</span><span class="params">(<span class="type">int</span> x)</span> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="type">int</span> max = <span class="number">1</span> << <span class="number">31</span> - <span class="number">1</span>;</span><br><span class="line"> <span class="type">int</span> ret = <span class="number">0</span>;</span><br><span class="line"> max = (max - <span class="number">1</span>) * <span class="number">2</span> + <span class="number">1</span>;</span><br><span class="line"> <span class="type">int</span> min = <span class="number">1</span> << <span class="number">31</span>;</span><br><span class="line"> <span class="keyword">if</span>(x < <span class="number">0</span>)</span><br><span class="line"> <span class="keyword">while</span>(x != <span class="number">0</span>){</span><br><span class="line"> <span class="keyword">if</span>(ret < (min - x % <span class="number">10</span>) / <span class="number">10</span>)</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line"> ret = ret * <span class="number">10</span> + x % <span class="number">10</span>;</span><br><span class="line"> x = x / <span class="number">10</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> <span class="keyword">while</span>(x != <span class="number">0</span>){</span><br><span class="line"> <span class="keyword">if</span>(ret > (max -x % <span class="number">10</span>) / <span class="number">10</span>)</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line"> ret = ret * <span class="number">10</span> + x % <span class="number">10</span>;</span><br><span class="line"> x = x / <span class="number">10</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> ret;</span><br><span class="line"> }</span><br><span class="line">};</span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>c++</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Tomcat 系列篇一-介绍下 connector</title>
|
|
<url>/2023/09/10/Tomcat-%E7%B3%BB%E5%88%97%E7%AF%87%E4%B8%80-%E4%BB%8B%E7%BB%8D%E4%B8%8B-connector/</url>
|
|
<content><![CDATA[<p>tomcat的主体架构里，connector 作为核心的连接器<br><img data-src="https://img.nicksxs.me/blog/pMttov.png"><br>这也是个架构的优化，将连接跟请求处理分开，可以适配各种连接协议<br>连接器的初始化逻辑，是在初始化 WebServer 的时候调用<br><code>org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory#getWebServer</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> WebServer <span class="title function_">getWebServer</span><span class="params">(ServletContextInitializer... initializers)</span> {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.disableMBeanRegistry) {</span><br><span class="line"> Registry.disableRegistry();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">Tomcat</span> <span class="variable">tomcat</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Tomcat</span>();</span><br><span class="line"> <span class="type">File</span> <span class="variable">baseDir</span> <span class="operator">=</span> <span class="built_in">this</span>.baseDirectory != <span class="literal">null</span> ? <span class="built_in">this</span>.baseDirectory : <span class="built_in">this</span>.createTempDir(<span class="string">"tomcat"</span>);</span><br><span class="line"> tomcat.setBaseDir(baseDir.getAbsolutePath());</span><br><span class="line"> <span class="comment">// 这里就是创建 Connector</span></span><br><span class="line"> <span class="type">Connector</span> <span class="variable">connector</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Connector</span>(<span class="built_in">this</span>.protocol);</span><br><span class="line"> connector.setThrowOnFailure(<span class="literal">true</span>);</span><br><span class="line"> tomcat.getService().addConnector(connector);</span><br><span class="line"> <span class="built_in">this</span>.customizeConnector(connector);</span><br><span class="line"> tomcat.setConnector(connector);</span><br></pre></td></tr></table></figure>
|
|
<p>而 connector 中最重要的就是 <code>ProtocolHandler</code> ，初始化代码中<br><code>org.apache.catalina.connector.Connector#Connector(java.lang.String)</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">Connector</span><span class="params">(String protocol)</span> {</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">apr</span> <span class="operator">=</span> AprStatus.getUseAprConnector() && AprStatus.isInstanceCreated()</span><br><span class="line"> && AprLifecycleListener.isAprAvailable();</span><br><span class="line"> <span class="type">ProtocolHandler</span> <span class="variable">p</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> p = ProtocolHandler.create(protocol, apr);</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> log.error(sm.getString(</span><br><span class="line"> <span class="string">"coyoteConnector.protocolHandlerInstantiationFailed"</span>), e);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (p != <span class="literal">null</span>) {</span><br><span class="line"> protocolHandler = p;</span><br><span class="line"> protocolHandlerClassName = protocolHandler.getClass().getName();</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> protocolHandler = <span class="literal">null</span>;</span><br><span class="line"> protocolHandlerClassName = protocol;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// Default for Connector depends on this system property</span></span><br><span class="line"> setThrowOnFailure(Boolean.getBoolean(<span class="string">"org.apache.catalina.startup.EXIT_ON_INIT_FAILURE"</span>));</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里就调用了<br><code>org.apache.coyote.ProtocolHandler#create</code><br>根据协议来生成对应的，我们这里默认就是<br><code>org.apache.coyote.http11.Http11NioProtocol</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> ProtocolHandler <span class="title function_">create</span><span class="params">(String protocol, <span class="type">boolean</span> apr)</span></span><br><span class="line"> <span class="keyword">throws</span> ClassNotFoundException, InstantiationException, IllegalAccessException,</span><br><span class="line"> IllegalArgumentException, InvocationTargetException, NoSuchMethodException, SecurityException {</span><br><span class="line"> <span class="keyword">if</span> (protocol == <span class="literal">null</span> || <span class="string">"HTTP/1.1"</span>.equals(protocol)</span><br><span class="line"> || (!apr && org.apache.coyote.http11.Http11NioProtocol.class.getName().equals(protocol))</span><br><span class="line"> || (apr && org.apache.coyote.http11.Http11AprProtocol.class.getName().equals(protocol))) {</span><br><span class="line"> <span class="keyword">if</span> (apr) {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">org</span>.apache.coyote.http11.Http11AprProtocol();</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">org</span>.apache.coyote.http11.Http11NioProtocol();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (<span class="string">"AJP/1.3"</span>.equals(protocol)</span><br><span class="line"> || (!apr && org.apache.coyote.ajp.AjpNioProtocol.class.getName().equals(protocol))</span><br><span class="line"> || (apr && org.apache.coyote.ajp.AjpAprProtocol.class.getName().equals(protocol))) {</span><br><span class="line"> <span class="keyword">if</span> (apr) {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">org</span>.apache.coyote.ajp.AjpAprProtocol();</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">org</span>.apache.coyote.ajp.AjpNioProtocol();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// Instantiate protocol handler</span></span><br><span class="line"> Class<?> clazz = Class.forName(protocol);</span><br><span class="line"> <span class="keyword">return</span> (ProtocolHandler) clazz.getConstructor().newInstance();</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>而这个初始化就主要做的是初始化 <code>EndPoint</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">Http11NioProtocol</span><span class="params">()</span> {</span><br><span class="line"> <span class="built_in">super</span>(<span class="keyword">new</span> <span class="title class_">NioEndpoint</span>());</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这个调用父类的方法是调用的<br><code>org.apache.coyote.http11.AbstractHttp11Protocol#AbstractHttp11Protocol</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">AbstractHttp11Protocol</span><span class="params">(AbstractEndpoint<S,?> endpoint)</span> {</span><br><span class="line"> <span class="built_in">super</span>(endpoint);</span><br><span class="line"> setConnectionTimeout(Constants.DEFAULT_CONNECTION_TIMEOUT);</span><br><span class="line"> ConnectionHandler<S> cHandler = <span class="keyword">new</span> <span class="title class_">ConnectionHandler</span><>(<span class="built_in">this</span>);</span><br><span class="line"> setHandler(cHandler);</span><br><span class="line"> getEndpoint().setHandler(cHandler);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>而后在 Tomcat 启动后，在启动 connector 的时候<br>是在<code>StandardService</code> 添加 connector 时，启动了 connector<br><code>org.apache.catalina.core.StandardService#addConnector</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">addConnector</span><span class="params">(Connector connector)</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">synchronized</span> (connectorsLock) {</span><br><span class="line"> connector.setService(<span class="built_in">this</span>);</span><br><span class="line"> Connector results[] = <span class="keyword">new</span> <span class="title class_">Connector</span>[connectors.length + <span class="number">1</span>];</span><br><span class="line"> System.arraycopy(connectors, <span class="number">0</span>, results, <span class="number">0</span>, connectors.length);</span><br><span class="line"> results[connectors.length] = connector;</span><br><span class="line"> connectors = results;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (getState().isAvailable()) {</span><br><span class="line"> connector.start();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (LifecycleException e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalArgumentException</span>(</span><br><span class="line"> sm.getString(<span class="string">"standardService.connector.startFailed"</span>, connector), e);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Report this property change to interested listeners</span></span><br><span class="line"> support.firePropertyChange(<span class="string">"connector"</span>, <span class="literal">null</span>, connector);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>而后就会调用到 <code>Connector</code> 的 <code>initInternal</code> 方法<br><code>org.apache.catalina.connector.Connector#initInternal</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">initInternal</span><span class="params">()</span> <span class="keyword">throws</span> LifecycleException {</span><br><span class="line"></span><br><span class="line"> <span class="built_in">super</span>.initInternal();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (protocolHandler == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">LifecycleException</span>(</span><br><span class="line"> sm.getString(<span class="string">"coyoteConnector.protocolHandlerInstantiationFailed"</span>));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Initialize adapter</span></span><br><span class="line"> adapter = <span class="keyword">new</span> <span class="title class_">CoyoteAdapter</span>(<span class="built_in">this</span>);</span><br><span class="line"> protocolHandler.setAdapter(adapter);</span><br><span class="line"> <span class="keyword">if</span> (service != <span class="literal">null</span>) {</span><br><span class="line"> protocolHandler.setUtilityExecutor(service.getServer().getUtilityExecutor());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Make sure parseBodyMethodsSet has a default</span></span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == parseBodyMethodsSet) {</span><br><span class="line"> setParseBodyMethods(getParseBodyMethods());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (protocolHandler.isAprRequired() && !AprStatus.isInstanceCreated()) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">LifecycleException</span>(sm.getString(<span class="string">"coyoteConnector.protocolHandlerNoAprListener"</span>,</span><br><span class="line"> getProtocolHandlerClassName()));</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (protocolHandler.isAprRequired() && !AprStatus.isAprAvailable()) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">LifecycleException</span>(sm.getString(<span class="string">"coyoteConnector.protocolHandlerNoAprLibrary"</span>,</span><br><span class="line"> getProtocolHandlerClassName()));</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (AprStatus.isAprAvailable() && AprStatus.getUseOpenSSL() &&</span><br><span class="line"> protocolHandler <span class="keyword">instanceof</span> AbstractHttp11JsseProtocol) {</span><br><span class="line"> AbstractHttp11JsseProtocol<?> jsseProtocolHandler =</span><br><span class="line"> (AbstractHttp11JsseProtocol<?>) protocolHandler;</span><br><span class="line"> <span class="keyword">if</span> (jsseProtocolHandler.isSSLEnabled() &&</span><br><span class="line"> jsseProtocolHandler.getSslImplementationName() == <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// OpenSSL is compatible with the JSSE configuration, so use it if APR is available</span></span><br><span class="line"> jsseProtocolHandler.setSslImplementationName(OpenSSLImplementation.class.getName());</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> protocolHandler.init();</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">LifecycleException</span>(</span><br><span class="line"> sm.getString(<span class="string">"coyoteConnector.protocolHandlerInitializationFailed"</span>), e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这里继续往下走就是 protocolHandler 的 init 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">init</span><span class="params">()</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="keyword">if</span> (getLog().isInfoEnabled()) {</span><br><span class="line"> getLog().info(sm.getString(<span class="string">"abstractProtocolHandler.init"</span>, getName()));</span><br><span class="line"> logPortOffset();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (oname == <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// Component not pre-registered so register it</span></span><br><span class="line"> oname = createObjectName();</span><br><span class="line"> <span class="keyword">if</span> (oname != <span class="literal">null</span>) {</span><br><span class="line"> Registry.getRegistry(<span class="literal">null</span>, <span class="literal">null</span>).registerComponent(<span class="built_in">this</span>, oname, <span class="literal">null</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.domain != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">ObjectName</span> <span class="variable">rgOname</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ObjectName</span>(domain + <span class="string">":type=GlobalRequestProcessor,name="</span> + getName());</span><br><span class="line"> <span class="built_in">this</span>.rgOname = rgOname;</span><br><span class="line"> Registry.getRegistry(<span class="literal">null</span>, <span class="literal">null</span>).registerComponent(</span><br><span class="line"> getHandler().getGlobal(), rgOname, <span class="literal">null</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">String</span> <span class="variable">endpointName</span> <span class="operator">=</span> getName();</span><br><span class="line"> endpoint.setName(endpointName.substring(<span class="number">1</span>, endpointName.length()-<span class="number">1</span>));</span><br><span class="line"> endpoint.setDomain(domain);</span><br><span class="line"></span><br><span class="line"> endpoint.init();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>看一下继承关系<br><img data-src="https://img.nicksxs.me/blog/fivtyf.png"><br>然后就看到这里调用了 endpoint.init() ,走的也是父类的初始化方法，<br><code>org.apache.tomcat.util.net.AbstractEndpoint#init</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">final</span> <span class="keyword">void</span> <span class="title function_">init</span><span class="params">()</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="keyword">if</span> (bindOnInit) {</span><br><span class="line"> bindWithCleanup();</span><br><span class="line"> bindState = BindState.BOUND_ON_INIT;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.domain != <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// Register endpoint (as ThreadPool - historical name)</span></span><br><span class="line"> oname = <span class="keyword">new</span> <span class="title class_">ObjectName</span>(domain + <span class="string">":type=ThreadPool,name=\""</span> + getName() + <span class="string">"\""</span>);</span><br><span class="line"> Registry.getRegistry(<span class="literal">null</span>, <span class="literal">null</span>).registerComponent(<span class="built_in">this</span>, oname, <span class="literal">null</span>);</span><br><span class="line"></span><br><span class="line"> <span class="type">ObjectName</span> <span class="variable">socketPropertiesOname</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ObjectName</span>(domain +</span><br><span class="line"> <span class="string">":type=SocketProperties,name=\""</span> + getName() + <span class="string">"\""</span>);</span><br><span class="line"> socketProperties.setObjectName(socketPropertiesOname);</span><br><span class="line"> Registry.getRegistry(<span class="literal">null</span>, <span class="literal">null</span>).registerComponent(socketProperties, socketPropertiesOname, <span class="literal">null</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (SSLHostConfig sslHostConfig : findSslHostConfigs()) {</span><br><span class="line"> registerJmx(sslHostConfig);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后接着调用了<br><code>org.apache.tomcat.util.net.AbstractEndpoint#bindWithCleanup</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">bindWithCleanup</span><span class="params">()</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> bind();</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable t) {</span><br><span class="line"> <span class="comment">// Ensure open sockets etc. are cleaned up if something goes</span></span><br><span class="line"> <span class="comment">// wrong during bind</span></span><br><span class="line"> ExceptionUtils.handleThrowable(t);</span><br><span class="line"> unbind();</span><br><span class="line"> <span class="keyword">throw</span> t;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里的 bind 方法调用的是<br><code>org.apache.tomcat.util.net.NioEndpoint#bind</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">bind</span><span class="params">()</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> initServerSocket();</span><br><span class="line"></span><br><span class="line"> setStopLatch(<span class="keyword">new</span> <span class="title class_">CountDownLatch</span>(<span class="number">1</span>));</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Initialize SSL if needed</span></span><br><span class="line"> initialiseSsl();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里的 initServerSocket是后面抽出来的，方便扩展，主要就是开启 ServerSocketChannel，绑定端口</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// Separated out to make it easier for folks that extend NioEndpoint to</span></span><br><span class="line"> <span class="comment">// implement custom [server]sockets</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">initServerSocket</span><span class="params">()</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="keyword">if</span> (getUseInheritedChannel()) {</span><br><span class="line"> <span class="comment">// Retrieve the channel provided by the OS</span></span><br><span class="line"> <span class="type">Channel</span> <span class="variable">ic</span> <span class="operator">=</span> System.inheritedChannel();</span><br><span class="line"> <span class="keyword">if</span> (ic <span class="keyword">instanceof</span> ServerSocketChannel) {</span><br><span class="line"> serverSock = (ServerSocketChannel) ic;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (serverSock == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalArgumentException</span>(sm.getString(<span class="string">"endpoint.init.bind.inherited"</span>));</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (getUnixDomainSocketPath() != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">SocketAddress</span> <span class="variable">sa</span> <span class="operator">=</span> JreCompat.getInstance().getUnixDomainSocketAddress(getUnixDomainSocketPath());</span><br><span class="line"> serverSock = JreCompat.getInstance().openUnixDomainServerSocketChannel();</span><br><span class="line"> serverSock.bind(sa, getAcceptCount());</span><br><span class="line"> <span class="keyword">if</span> (getUnixDomainSocketPathPermissions() != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">Path</span> <span class="variable">path</span> <span class="operator">=</span> Paths.get(getUnixDomainSocketPath());</span><br><span class="line"> Set<PosixFilePermission> permissions =</span><br><span class="line"> PosixFilePermissions.fromString(getUnixDomainSocketPathPermissions());</span><br><span class="line"> <span class="keyword">if</span> (path.getFileSystem().supportedFileAttributeViews().contains(<span class="string">"posix"</span>)) {</span><br><span class="line"> FileAttribute<Set<PosixFilePermission>> attrs = PosixFilePermissions.asFileAttribute(permissions);</span><br><span class="line"> Files.setAttribute(path, attrs.name(), attrs.value());</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> java.io.<span class="type">File</span> <span class="variable">file</span> <span class="operator">=</span> path.toFile();</span><br><span class="line"> <span class="keyword">if</span> (permissions.contains(PosixFilePermission.OTHERS_READ) && !file.setReadable(<span class="literal">true</span>, <span class="literal">false</span>)) {</span><br><span class="line"> log.warn(sm.getString(<span class="string">"endpoint.nio.perms.readFail"</span>, file.getPath()));</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (permissions.contains(PosixFilePermission.OTHERS_WRITE) && !file.setWritable(<span class="literal">true</span>, <span class="literal">false</span>)) {</span><br><span class="line"> log.warn(sm.getString(<span class="string">"endpoint.nio.perms.writeFail"</span>, file.getPath()));</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> serverSock = ServerSocketChannel.open();</span><br><span class="line"> socketProperties.setProperties(serverSock.socket());</span><br><span class="line"> <span class="type">InetSocketAddress</span> <span class="variable">addr</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">InetSocketAddress</span>(getAddress(), getPortWithOffset());</span><br><span class="line"> serverSock.bind(addr, getAcceptCount());</span><br><span class="line"> }</span><br><span class="line"> serverSock.configureBlocking(<span class="literal">true</span>); <span class="comment">//mimic APR behavior</span></span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>接着我们来看下 start 方法，这里多数是复用的 父类的方法<br><code>org.apache.tomcat.util.net.AbstractEndpoint#start</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">final</span> <span class="keyword">void</span> <span class="title function_">start</span><span class="params">()</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="keyword">if</span> (bindState == BindState.UNBOUND) {</span><br><span class="line"> bindWithCleanup();</span><br><span class="line"> bindState = BindState.BOUND_ON_START;</span><br><span class="line"> }</span><br><span class="line"> startInternal();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>startInternal 才是 NioEndPoint 中的处理</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Start the NIO endpoint, creating acceptor, poller threads.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">startInternal</span><span class="params">()</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!running) {</span><br><span class="line"> running = <span class="literal">true</span>;</span><br><span class="line"> paused = <span class="literal">false</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (socketProperties.getProcessorCache() != <span class="number">0</span>) {</span><br><span class="line"> processorCache = <span class="keyword">new</span> <span class="title class_">SynchronizedStack</span><>(SynchronizedStack.DEFAULT_SIZE,</span><br><span class="line"> socketProperties.getProcessorCache());</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (socketProperties.getEventCache() != <span class="number">0</span>) {</span><br><span class="line"> eventCache = <span class="keyword">new</span> <span class="title class_">SynchronizedStack</span><>(SynchronizedStack.DEFAULT_SIZE,</span><br><span class="line"> socketProperties.getEventCache());</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (socketProperties.getBufferPool() != <span class="number">0</span>) {</span><br><span class="line"> nioChannels = <span class="keyword">new</span> <span class="title class_">SynchronizedStack</span><>(SynchronizedStack.DEFAULT_SIZE,</span><br><span class="line"> socketProperties.getBufferPool());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Create worker collection</span></span><br><span class="line"> <span class="keyword">if</span> (getExecutor() == <span class="literal">null</span>) {</span><br><span class="line"> createExecutor();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> initializeConnectionLatch();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Start poller thread</span></span><br><span class="line"> poller = <span class="keyword">new</span> <span class="title class_">Poller</span>();</span><br><span class="line"> <span class="type">Thread</span> <span class="variable">pollerThread</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Thread</span>(poller, getName() + <span class="string">"-Poller"</span>);</span><br><span class="line"> pollerThread.setPriority(threadPriority);</span><br><span class="line"> pollerThread.setDaemon(<span class="literal">true</span>);</span><br><span class="line"> pollerThread.start();</span><br><span class="line"></span><br><span class="line"> startAcceptorThread();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>上面是启动了一个 Poller 线程，在<code>startAcceptorThread</code> 里是启动了 acceptor</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">startAcceptorThread</span><span class="params">()</span> {</span><br><span class="line"> acceptor = <span class="keyword">new</span> <span class="title class_">Acceptor</span><>(<span class="built_in">this</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">threadName</span> <span class="operator">=</span> getName() + <span class="string">"-Acceptor"</span>;</span><br><span class="line"> acceptor.setThreadName(threadName);</span><br><span class="line"> <span class="type">Thread</span> <span class="variable">t</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Thread</span>(acceptor, threadName);</span><br><span class="line"> t.setPriority(getAcceptorThreadPriority());</span><br><span class="line"> t.setDaemon(getDaemon());</span><br><span class="line"> t.start();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>启动后运行的代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"></span><br><span class="line"> <span class="type">int</span> <span class="variable">errorDelay</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// Loop until we receive a shutdown command</span></span><br><span class="line"> <span class="keyword">while</span> (!stopCalled) {</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Loop if endpoint is paused</span></span><br><span class="line"> <span class="keyword">while</span> (endpoint.isPaused() && !stopCalled) {</span><br><span class="line"> state = AcceptorState.PAUSED;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> Thread.sleep(<span class="number">50</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> <span class="comment">// Ignore</span></span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (stopCalled) {</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> state = AcceptorState.RUNNING;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">//if we have reached max connections, wait</span></span><br><span class="line"> endpoint.countUpOrAwaitConnection();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Endpoint might have been paused while waiting for latch</span></span><br><span class="line"> <span class="comment">// If that is the case, don't accept new connections</span></span><br><span class="line"> <span class="keyword">if</span> (endpoint.isPaused()) {</span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">U</span> <span class="variable">socket</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// Accept the next incoming connection from the server</span></span><br><span class="line"> <span class="comment">// socket</span></span><br><span class="line"> socket = endpoint.serverSocketAccept();</span><br><span class="line"> } <span class="keyword">catch</span> (Exception ioe) {</span><br><span class="line"> <span class="comment">// We didn't get a socket</span></span><br><span class="line"> endpoint.countDownConnection();</span><br><span class="line"> <span class="keyword">if</span> (endpoint.isRunning()) {</span><br><span class="line"> <span class="comment">// Introduce delay if necessary</span></span><br><span class="line"> errorDelay = handleExceptionWithDelay(errorDelay);</span><br><span class="line"> <span class="comment">// re-throw</span></span><br><span class="line"> <span class="keyword">throw</span> ioe;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// Successful accept, reset the error delay</span></span><br><span class="line"> errorDelay = <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Configure the socket</span></span><br><span class="line"> <span class="keyword">if</span> (!stopCalled && !endpoint.isPaused()) {</span><br><span class="line"> <span class="comment">// setSocketOptions() will hand the socket off to</span></span><br><span class="line"> <span class="comment">// an appropriate processor if successful</span></span><br><span class="line"> <span class="keyword">if</span> (!endpoint.setSocketOptions(socket)) {</span><br><span class="line"> endpoint.closeSocket(socket);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> endpoint.destroySocket(socket);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable t) {</span><br><span class="line"> ExceptionUtils.handleThrowable(t);</span><br><span class="line"> <span class="type">String</span> <span class="variable">msg</span> <span class="operator">=</span> sm.getString(<span class="string">"endpoint.accept.fail"</span>);</span><br><span class="line"> <span class="comment">// APR specific.</span></span><br><span class="line"> <span class="comment">// Could push this down but not sure it is worth the trouble.</span></span><br><span class="line"> <span class="keyword">if</span> (t <span class="keyword">instanceof</span> Error) {</span><br><span class="line"> <span class="type">Error</span> <span class="variable">e</span> <span class="operator">=</span> (Error) t;</span><br><span class="line"> <span class="keyword">if</span> (e.getError() == <span class="number">233</span>) {</span><br><span class="line"> <span class="comment">// Not an error on HP-UX so log as a warning</span></span><br><span class="line"> <span class="comment">// so it can be filtered out on that platform</span></span><br><span class="line"> <span class="comment">// See bug 50273</span></span><br><span class="line"> log.warn(msg, t);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> log.error(msg, t);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> log.error(msg, t);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> stopLatch.countDown();</span><br><span class="line"> }</span><br><span class="line"> state = AcceptorState.ENDED;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这行<code>socket = endpoint.serverSocketAccept();</code>是 accept 等待线程进来，进来以后调用 </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="type">boolean</span> <span class="title function_">setSocketOptions</span><span class="params">(SocketChannel socket)</span> {</span><br><span class="line"> <span class="type">NioSocketWrapper</span> <span class="variable">socketWrapper</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// Allocate channel and wrapper</span></span><br><span class="line"> <span class="type">NioChannel</span> <span class="variable">channel</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">if</span> (nioChannels != <span class="literal">null</span>) {</span><br><span class="line"> channel = nioChannels.pop();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (channel == <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">SocketBufferHandler</span> <span class="variable">bufhandler</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SocketBufferHandler</span>(</span><br><span class="line"> socketProperties.getAppReadBufSize(),</span><br><span class="line"> socketProperties.getAppWriteBufSize(),</span><br><span class="line"> socketProperties.getDirectBuffer());</span><br><span class="line"> <span class="keyword">if</span> (isSSLEnabled()) {</span><br><span class="line"> channel = <span class="keyword">new</span> <span class="title class_">SecureNioChannel</span>(bufhandler, <span class="built_in">this</span>);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> channel = <span class="keyword">new</span> <span class="title class_">NioChannel</span>(bufhandler);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="type">NioSocketWrapper</span> <span class="variable">newWrapper</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">NioSocketWrapper</span>(channel, <span class="built_in">this</span>);</span><br><span class="line"> channel.reset(socket, newWrapper);</span><br><span class="line"> connections.put(socket, newWrapper);</span><br><span class="line"> socketWrapper = newWrapper;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Set socket properties</span></span><br><span class="line"> <span class="comment">// Disable blocking, polling will be used</span></span><br><span class="line"> socket.configureBlocking(<span class="literal">false</span>);</span><br><span class="line"> <span class="keyword">if</span> (getUnixDomainSocketPath() == <span class="literal">null</span>) {</span><br><span class="line"> socketProperties.setProperties(socket.socket());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> socketWrapper.setReadTimeout(getConnectionTimeout());</span><br><span class="line"> socketWrapper.setWriteTimeout(getConnectionTimeout());</span><br><span class="line"> socketWrapper.setKeepAliveLeft(NioEndpoint.<span class="built_in">this</span>.getMaxKeepAliveRequests());</span><br><span class="line"> poller.register(socketWrapper);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable t) {</span><br><span class="line"> ExceptionUtils.handleThrowable(t);</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> log.error(sm.getString(<span class="string">"endpoint.socketOptionsError"</span>), t);</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable tt) {</span><br><span class="line"> ExceptionUtils.handleThrowable(tt);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (socketWrapper == <span class="literal">null</span>) {</span><br><span class="line"> destroySocket(socket);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// Tell to close the socket if needed</span></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里就是最重要的封装了 <code>NioSocketWrapper</code>, 然后注册到 Poller，<br>我们再来看 Poller 代码，注册其实是添加事件 event</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">register</span><span class="params">(<span class="keyword">final</span> NioSocketWrapper socketWrapper)</span> {</span><br><span class="line"> socketWrapper.interestOps(SelectionKey.OP_READ);<span class="comment">//this is what OP_REGISTER turns into.</span></span><br><span class="line"> <span class="type">PollerEvent</span> <span class="variable">event</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">if</span> (eventCache != <span class="literal">null</span>) {</span><br><span class="line"> event = eventCache.pop();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (event == <span class="literal">null</span>) {</span><br><span class="line"> event = <span class="keyword">new</span> <span class="title class_">PollerEvent</span>(socketWrapper, OP_REGISTER);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> event.reset(socketWrapper, OP_REGISTER);</span><br><span class="line"> }</span><br><span class="line"> addEvent(event);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后Poller 的运行方法会处理这些 event</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> <span class="comment">// Loop until destroy() is called</span></span><br><span class="line"> <span class="keyword">while</span> (<span class="literal">true</span>) {</span><br><span class="line"></span><br><span class="line"> <span class="type">boolean</span> <span class="variable">hasEvents</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (!close) {</span><br><span class="line"> hasEvents = events();</span><br><span class="line"> <span class="keyword">if</span> (wakeupCounter.getAndSet(-<span class="number">1</span>) > <span class="number">0</span>) {</span><br><span class="line"> <span class="comment">// If we are here, means we have other stuff to do</span></span><br><span class="line"> <span class="comment">// Do a non blocking select</span></span><br><span class="line"> keyCount = selector.selectNow();</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> keyCount = selector.select(selectorTimeout);</span><br><span class="line"> }</span><br><span class="line"> wakeupCounter.set(<span class="number">0</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (close) {</span><br><span class="line"> events();</span><br><span class="line"> timeout(<span class="number">0</span>, <span class="literal">false</span>);</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> selector.close();</span><br><span class="line"> } <span class="keyword">catch</span> (IOException ioe) {</span><br><span class="line"> log.error(sm.getString(<span class="string">"endpoint.nio.selectorCloseFail"</span>), ioe);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// Either we timed out or we woke up, process events first</span></span><br><span class="line"> <span class="keyword">if</span> (keyCount == <span class="number">0</span>) {</span><br><span class="line"> hasEvents = (hasEvents | events());</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable x) {</span><br><span class="line"> ExceptionUtils.handleThrowable(x);</span><br><span class="line"> log.error(sm.getString(<span class="string">"endpoint.nio.selectorLoopError"</span>), x);</span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> Iterator<SelectionKey> iterator =</span><br><span class="line"> keyCount > <span class="number">0</span> ? selector.selectedKeys().iterator() : <span class="literal">null</span>;</span><br><span class="line"> <span class="comment">// Walk through the collection of ready keys and dispatch</span></span><br><span class="line"> <span class="comment">// any active event.</span></span><br><span class="line"> <span class="keyword">while</span> (iterator != <span class="literal">null</span> && iterator.hasNext()) {</span><br><span class="line"> <span class="type">SelectionKey</span> <span class="variable">sk</span> <span class="operator">=</span> iterator.next();</span><br><span class="line"> iterator.remove();</span><br><span class="line"> <span class="type">NioSocketWrapper</span> <span class="variable">socketWrapper</span> <span class="operator">=</span> (NioSocketWrapper) sk.attachment();</span><br><span class="line"> <span class="comment">// Attachment may be null if another thread has called</span></span><br><span class="line"> <span class="comment">// cancelledKey()</span></span><br><span class="line"> <span class="keyword">if</span> (socketWrapper != <span class="literal">null</span>) {</span><br><span class="line"> processKey(sk, socketWrapper);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Process timeouts</span></span><br><span class="line"> timeout(keyCount,hasEvents);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> getStopLatch().countDown();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>如果 events 方法返回了 true 代表有事件，就会跑到<code>processKey(sk, socketWrapper);</code> 来处理这个事件<br>而这里的 processKey 也比较复杂，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">processKey</span><span class="params">(SelectionKey sk, NioSocketWrapper socketWrapper)</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (close) {</span><br><span class="line"> cancelledKey(sk, socketWrapper);</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (sk.isValid()) {</span><br><span class="line"> <span class="keyword">if</span> (sk.isReadable() || sk.isWritable()) {</span><br><span class="line"> <span class="keyword">if</span> (socketWrapper.getSendfileData() != <span class="literal">null</span>) {</span><br><span class="line"> processSendfile(sk, socketWrapper, <span class="literal">false</span>);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> unreg(sk, socketWrapper, sk.readyOps());</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">closeSocket</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="comment">// Read goes before write</span></span><br><span class="line"> <span class="keyword">if</span> (sk.isReadable()) {</span><br><span class="line"> <span class="keyword">if</span> (socketWrapper.readOperation != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">if</span> (!socketWrapper.readOperation.process()) {</span><br><span class="line"> closeSocket = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (socketWrapper.readBlocking) {</span><br><span class="line"> <span class="keyword">synchronized</span> (socketWrapper.readLock) {</span><br><span class="line"> socketWrapper.readBlocking = <span class="literal">false</span>;</span><br><span class="line"> socketWrapper.readLock.notify();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (!processSocket(socketWrapper, SocketEvent.OPEN_READ, <span class="literal">true</span>)) {</span><br><span class="line"> closeSocket = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (!closeSocket && sk.isWritable()) {</span><br><span class="line"> <span class="keyword">if</span> (socketWrapper.writeOperation != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">if</span> (!socketWrapper.writeOperation.process()) {</span><br><span class="line"> closeSocket = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (socketWrapper.writeBlocking) {</span><br><span class="line"> <span class="keyword">synchronized</span> (socketWrapper.writeLock) {</span><br><span class="line"> socketWrapper.writeBlocking = <span class="literal">false</span>;</span><br><span class="line"> socketWrapper.writeLock.notify();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (!processSocket(socketWrapper, SocketEvent.OPEN_WRITE, <span class="literal">true</span>)) {</span><br><span class="line"> closeSocket = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (closeSocket) {</span><br><span class="line"> cancelledKey(sk, socketWrapper);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// Invalid key</span></span><br><span class="line"> cancelledKey(sk, socketWrapper);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (CancelledKeyException ckx) {</span><br><span class="line"> cancelledKey(sk, socketWrapper);</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable t) {</span><br><span class="line"> ExceptionUtils.handleThrowable(t);</span><br><span class="line"> log.error(sm.getString(<span class="string">"endpoint.nio.keyProcessingError"</span>), t);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>正常请求回到这<br><code>else if (!processSocket(socketWrapper, SocketEvent.OPEN_READ, true)) </code><br>然后调用<code>processSocket</code> 进行处理，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">processSocket</span><span class="params">(SocketWrapperBase<S> socketWrapper,</span></span><br><span class="line"><span class="params"> SocketEvent event, <span class="type">boolean</span> dispatch)</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (socketWrapper == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> SocketProcessorBase<S> sc = <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">if</span> (processorCache != <span class="literal">null</span>) {</span><br><span class="line"> sc = processorCache.pop();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (sc == <span class="literal">null</span>) {</span><br><span class="line"> sc = createSocketProcessor(socketWrapper, event);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> sc.reset(socketWrapper, event);</span><br><span class="line"> }</span><br><span class="line"> <span class="type">Executor</span> <span class="variable">executor</span> <span class="operator">=</span> getExecutor();</span><br><span class="line"> <span class="keyword">if</span> (dispatch && executor != <span class="literal">null</span>) {</span><br><span class="line"> executor.execute(sc);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> sc.run();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (RejectedExecutionException ree) {</span><br><span class="line"> getLog().warn(sm.getString(<span class="string">"endpoint.executor.fail"</span>, socketWrapper) , ree);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable t) {</span><br><span class="line"> ExceptionUtils.handleThrowable(t);</span><br><span class="line"> <span class="comment">// This means we got an OOM or similar creating a thread, or that</span></span><br><span class="line"> <span class="comment">// the pool and its queue are full</span></span><br><span class="line"> getLog().error(sm.getString(<span class="string">"endpoint.process.fail"</span>), t);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里就会调用 <code>createSocketProcessor</code> 进行处理了，不过这是下一篇的内容了。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
<category>Tomcat</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Tomcat</tag>
|
|
<tag>SpringBoot</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Tomcat 系列篇七-介绍下 Filter 注册逻辑</title>
|
|
<url>/2023/10/29/Tomcat-%E7%B3%BB%E5%88%97%E7%AF%87%E4%B8%83-%E4%BB%8B%E7%BB%8D%E4%B8%8B-Filter-%E6%B3%A8%E5%86%8C%E9%80%BB%E8%BE%91/</url>
|
|
<content><![CDATA[<p>在<br>org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext#createWebServer 的时候会调用</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="built_in">this</span>.webServer = factory.getWebServer(getSelfInitializer());</span><br></pre></td></tr></table></figure>
|
|
<p>就会调用到之前说到的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> org.springframework.boot.web.servlet.ServletContextInitializer <span class="title function_">getSelfInitializer</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>::selfInitialize;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">selfInitialize</span><span class="params">(ServletContext servletContext)</span> <span class="keyword">throws</span> ServletException {</span><br><span class="line"> prepareWebApplicationContext(servletContext);</span><br><span class="line"> registerApplicationScope(servletContext);</span><br><span class="line"> WebApplicationContextUtils.registerEnvironmentBeans(getBeanFactory(), servletContext);</span><br><span class="line"> <span class="keyword">for</span> (ServletContextInitializer beans : getServletContextInitializerBeans()) {</span><br><span class="line"> beans.onStartup(servletContext);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>而这里有个<br><code>org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext#getServletContextInitializerBeans</code><br>获取初始化所需要的 bean</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> Collection<ServletContextInitializer> <span class="title function_">getServletContextInitializerBeans</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">ServletContextInitializerBeans</span>(getBeanFactory());</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>里面就是 new 了这个 <code>org.springframework.boot.web.servlet.ServletContextInitializerBeans#ServletContextInitializerBeans</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">ServletContextInitializerBeans</span><span class="params">(ListableBeanFactory beanFactory,</span></span><br><span class="line"><span class="params"> Class<? extends ServletContextInitializer>... initializerTypes)</span> {</span><br><span class="line"> <span class="built_in">this</span>.initializers = <span class="keyword">new</span> <span class="title class_">LinkedMultiValueMap</span><>();</span><br><span class="line"> <span class="built_in">this</span>.initializerTypes = (initializerTypes.length != <span class="number">0</span>) ? Arrays.asList(initializerTypes)</span><br><span class="line"> : Collections.singletonList(ServletContextInitializer.class);</span><br><span class="line"> <span class="comment">// 这里是添加 ServletContextInitializer</span></span><br><span class="line"> addServletContextInitializerBeans(beanFactory);</span><br><span class="line"> <span class="comment">// filter 是在这里</span></span><br><span class="line"> addAdaptableBeans(beanFactory);</span><br><span class="line"> List<ServletContextInitializer> sortedInitializers = <span class="built_in">this</span>.initializers.values().stream()</span><br><span class="line"> .flatMap((value) -> value.stream().sorted(AnnotationAwareOrderComparator.INSTANCE))</span><br><span class="line"> .collect(Collectors.toList());</span><br><span class="line"> <span class="built_in">this</span>.sortedList = Collections.unmodifiableList(sortedInitializers);</span><br><span class="line"> logMappings(<span class="built_in">this</span>.initializers);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>具体来看看 <code>org.springframework.boot.web.servlet.ServletContextInitializerBeans#addAdaptableBeans</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">addAdaptableBeans</span><span class="params">(ListableBeanFactory beanFactory)</span> {</span><br><span class="line"> <span class="type">MultipartConfigElement</span> <span class="variable">multipartConfig</span> <span class="operator">=</span> getMultipartConfig(beanFactory);</span><br><span class="line"> <span class="comment">// 这里把 Servlet 作为 RegistrationBean</span></span><br><span class="line"> addAsRegistrationBean(beanFactory, Servlet.class, <span class="keyword">new</span> <span class="title class_">ServletRegistrationBeanAdapter</span>(multipartConfig));</span><br><span class="line"> <span class="comment">// 这里把 Filter 作为 RegistrationBean</span></span><br><span class="line"> addAsRegistrationBean(beanFactory, Filter.class, <span class="keyword">new</span> <span class="title class_">FilterRegistrationBeanAdapter</span>());</span><br><span class="line"> <span class="keyword">for</span> (Class<?> listenerType : ServletListenerRegistrationBean.getSupportedTypes()) {</span><br><span class="line"> addAsRegistrationBean(beanFactory, EventListener.class, (Class<EventListener>) listenerType,</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">ServletListenerRegistrationBeanAdapter</span>());</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>里面是这两段逻辑，先从 beanfactory 里获取到该类型的 bean，然后包装成 RegistrationBean，添加到 initializers</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <T> <span class="keyword">void</span> <span class="title function_">addAsRegistrationBean</span><span class="params">(ListableBeanFactory beanFactory, Class<T> type,</span></span><br><span class="line"><span class="params"> RegistrationBeanAdapter<T> adapter)</span> {</span><br><span class="line"> addAsRegistrationBean(beanFactory, type, type, adapter);</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">private</span> <T, B <span class="keyword">extends</span> <span class="title class_">T</span>> <span class="keyword">void</span> <span class="title function_">addAsRegistrationBean</span><span class="params">(ListableBeanFactory beanFactory, Class<T> type,</span></span><br><span class="line"><span class="params"> Class<B> beanType, RegistrationBeanAdapter<T> adapter)</span> {</span><br><span class="line"> List<Map.Entry<String, B>> entries = getOrderedBeansOfType(beanFactory, beanType, <span class="built_in">this</span>.seen);</span><br><span class="line"> <span class="keyword">for</span> (Entry<String, B> entry : entries) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">beanName</span> <span class="operator">=</span> entry.getKey();</span><br><span class="line"> <span class="type">B</span> <span class="variable">bean</span> <span class="operator">=</span> entry.getValue();</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.seen.add(bean)) {</span><br><span class="line"> <span class="comment">// One that we haven't already seen</span></span><br><span class="line"> <span class="type">RegistrationBean</span> <span class="variable">registration</span> <span class="operator">=</span> adapter.createRegistrationBean(beanName, bean, entries.size());</span><br><span class="line"> <span class="type">int</span> <span class="variable">order</span> <span class="operator">=</span> getOrder(bean);</span><br><span class="line"> registration.setOrder(order);</span><br><span class="line"> <span class="built_in">this</span>.initializers.add(type, registration);</span><br><span class="line"> <span class="keyword">if</span> (logger.isTraceEnabled()) {</span><br><span class="line"> logger.trace(<span class="string">"Created "</span> + type.getSimpleName() + <span class="string">" initializer for bean '"</span> + beanName + <span class="string">"'; order="</span></span><br><span class="line"> + order + <span class="string">", resource="</span> + getResourceDescription(beanName, beanFactory));</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后再回到第一段，其中的循环里面，因为 ServletContextInitializerBeans 继承了 AbstractCollection 并且实现了 iterator 接口</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">for</span> (ServletContextInitializer beans : getServletContextInitializerBeans()) {</span><br><span class="line"> beans.onStartup(servletContext);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>在 for 循环中会把 Initializer 排序后的 sortedList 返回 iterator，也就是前面把 filter 包装成的 RegistrationBean</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Iterator<ServletContextInitializer> <span class="title function_">iterator</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.sortedList.iterator();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后调用了 onStartup 方法，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">final</span> <span class="keyword">void</span> <span class="title function_">onStartup</span><span class="params">(ServletContext servletContext)</span> <span class="keyword">throws</span> ServletException {</span><br><span class="line"> <span class="type">String</span> <span class="variable">description</span> <span class="operator">=</span> getDescription();</span><br><span class="line"> <span class="keyword">if</span> (!isEnabled()) {</span><br><span class="line"> logger.info(StringUtils.capitalize(description) + <span class="string">" was not registered (disabled)"</span>);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> register(description, servletContext);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>就会去执行注册逻辑</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
<category>Tomcat</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Tomcat</tag>
|
|
<tag>SpringBoot</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Tomcat 系列篇三-介绍下 Coyote</title>
|
|
<url>/2023/09/24/Tomcat-%E7%B3%BB%E5%88%97%E7%AF%87%E4%B8%89-%E4%BB%8B%E7%BB%8D%E4%B8%8B-Coyote/</url>
|
|
<content><![CDATA[<p>前面介绍了 connector，这里边还有个很重要的概念是 Coyote，真正将前面的 connector 跟后面的 container 做了连接，<br><code>org.apache.tomcat.util.net.AbstractEndpoint#createSocketProcessor</code> 从这里开始<br>然后会调用到 <code>org.apache.tomcat.util.net.NioEndpoint#createSocketProcessor</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">protected</span> SocketProcessorBase<NioChannel> <span class="title function_">createSocketProcessor</span><span class="params">(</span></span><br><span class="line"><span class="params"> SocketWrapperBase<NioChannel> socketWrapper, SocketEvent event)</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">SocketProcessor</span>(socketWrapper, event);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>里面实际的是 new 了<br> <code>org.apache.tomcat.util.net.NioEndpoint.SocketProcessor#SocketProcessor</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">class</span> <span class="title class_">SocketProcessor</span> <span class="keyword">extends</span> <span class="title class_">SocketProcessorBase</span><NioChannel> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">SocketProcessor</span><span class="params">(SocketWrapperBase<NioChannel> socketWrapper, SocketEvent event)</span> {</span><br><span class="line"> <span class="built_in">super</span>(socketWrapper, event);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">doRun</span><span class="params">()</span> {</span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> * Do not cache and re-use the value of socketWrapper.getSocket() in</span></span><br><span class="line"><span class="comment"> * this method. If the socket closes the value will be updated to</span></span><br><span class="line"><span class="comment"> * CLOSED_NIO_CHANNEL and the previous value potentially re-used for</span></span><br><span class="line"><span class="comment"> * a new connection. That can result in a stale cached value which</span></span><br><span class="line"><span class="comment"> * in turn can result in unintentionally closing currently active</span></span><br><span class="line"><span class="comment"> * connections.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="type">Poller</span> <span class="variable">poller</span> <span class="operator">=</span> NioEndpoint.<span class="built_in">this</span>.poller;</span><br><span class="line"> <span class="keyword">if</span> (poller == <span class="literal">null</span>) {</span><br><span class="line"> socketWrapper.close();</span><br></pre></td></tr></table></figure>
|
|
<p>然后是 <code>org.apache.tomcat.util.net.NioEndpoint.SocketProcessor#doRun</code> 这里开始运行</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">doRun</span><span class="params">()</span> {</span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> * Do not cache and re-use the value of socketWrapper.getSocket() in</span></span><br><span class="line"><span class="comment"> * this method. If the socket closes the value will be updated to</span></span><br><span class="line"><span class="comment"> * CLOSED_NIO_CHANNEL and the previous value potentially re-used for</span></span><br><span class="line"><span class="comment"> * a new connection. That can result in a stale cached value which</span></span><br><span class="line"><span class="comment"> * in turn can result in unintentionally closing currently active</span></span><br><span class="line"><span class="comment"> * connections.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="type">Poller</span> <span class="variable">poller</span> <span class="operator">=</span> NioEndpoint.<span class="built_in">this</span>.poller;</span><br><span class="line"> <span class="keyword">if</span> (poller == <span class="literal">null</span>) {</span><br><span class="line"> socketWrapper.close();</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">int</span> <span class="variable">handshake</span> <span class="operator">=</span> -<span class="number">1</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (socketWrapper.getSocket().isHandshakeComplete()) {</span><br><span class="line"> <span class="comment">// No TLS handshaking required. Let the handler</span></span><br><span class="line"> <span class="comment">// process this socket / event combination.</span></span><br><span class="line"> handshake = <span class="number">0</span>;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (event == SocketEvent.STOP || event == SocketEvent.DISCONNECT ||</span><br><span class="line"> event == SocketEvent.ERROR) {</span><br><span class="line"> <span class="comment">// Unable to complete the TLS handshake. Treat it as</span></span><br><span class="line"> <span class="comment">// if the handshake failed.</span></span><br><span class="line"> handshake = -<span class="number">1</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> handshake = socketWrapper.getSocket().handshake(event == SocketEvent.OPEN_READ, event == SocketEvent.OPEN_WRITE);</span><br><span class="line"> <span class="comment">// The handshake process reads/writes from/to the</span></span><br><span class="line"> <span class="comment">// socket. status may therefore be OPEN_WRITE once</span></span><br><span class="line"> <span class="comment">// the handshake completes. However, the handshake</span></span><br><span class="line"> <span class="comment">// happens when the socket is opened so the status</span></span><br><span class="line"> <span class="comment">// must always be OPEN_READ after it completes. It</span></span><br><span class="line"> <span class="comment">// is OK to always set this as it is only used if</span></span><br><span class="line"> <span class="comment">// the handshake completes.</span></span><br><span class="line"> event = SocketEvent.OPEN_READ;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (IOException x) {</span><br><span class="line"> handshake = -<span class="number">1</span>;</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"Error during SSL handshake"</span>,x);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (CancelledKeyException ckx) {</span><br><span class="line"> handshake = -<span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (handshake == <span class="number">0</span>) {</span><br><span class="line"> <span class="type">SocketState</span> <span class="variable">state</span> <span class="operator">=</span> SocketState.OPEN;</span><br><span class="line"> <span class="comment">// Process the request from this socket</span></span><br><span class="line"> <span class="keyword">if</span> (event == <span class="literal">null</span>) {</span><br><span class="line"> state = getHandler().process(socketWrapper, SocketEvent.OPEN_READ);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> state = getHandler().process(socketWrapper, event);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (state == SocketState.CLOSED) {</span><br><span class="line"> poller.cancelledKey(getSelectionKey(), socketWrapper);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p><code>org.apache.coyote.AbstractProtocol.ConnectionHandler#process</code> 这个 <code>getHandler</code> 是哪个呢</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">AbstractHttp11Protocol</span><span class="params">(AbstractEndpoint<S,?> endpoint)</span> {</span><br><span class="line"> <span class="built_in">super</span>(endpoint);</span><br><span class="line"> setConnectionTimeout(Constants.DEFAULT_CONNECTION_TIMEOUT);</span><br><span class="line"> ConnectionHandler<S> cHandler = <span class="keyword">new</span> <span class="title class_">ConnectionHandler</span><>(<span class="built_in">this</span>);</span><br><span class="line"> setHandler(cHandler);</span><br><span class="line"> getEndpoint().setHandler(cHandler);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>上面补充下这个 Handler，帮助后面的理解，而这个 connectionHandler 则是实现了 <code>AbstractEndpoint.Handler</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">static</span> <span class="keyword">class</span> <span class="title class_">ConnectionHandler</span><S> <span class="keyword">implements</span> <span class="title class_">AbstractEndpoint</span>.Handler<S> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> AbstractProtocol<S> proto;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="type">RequestGroupInfo</span> <span class="variable">global</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">RequestGroupInfo</span>();</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="type">AtomicLong</span> <span class="variable">registerCount</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">AtomicLong</span>(<span class="number">0</span>);</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="type">RecycledProcessors</span> <span class="variable">recycledProcessors</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">RecycledProcessors</span>(<span class="built_in">this</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">ConnectionHandler</span><span class="params">(AbstractProtocol<S> proto)</span> {</span><br><span class="line"> <span class="built_in">this</span>.proto = proto;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> AbstractProtocol<S> <span class="title function_">getProtocol</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> proto;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后会继续寻找真实的 Processer</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> SocketState <span class="title function_">process</span><span class="params">(SocketWrapperBase<S> wrapper, SocketEvent status)</span> {</span><br><span class="line"> <span class="keyword">if</span> (getLog().isDebugEnabled()) {</span><br><span class="line"> getLog().debug(sm.getString(<span class="string">"abstractConnectionHandler.process"</span>,</span><br><span class="line"> wrapper.getSocket(), status));</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (wrapper == <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// Nothing to do. Socket has been closed.</span></span><br><span class="line"> <span class="keyword">return</span> SocketState.CLOSED;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">S</span> <span class="variable">socket</span> <span class="operator">=</span> wrapper.getSocket();</span><br><span class="line"></span><br><span class="line"> <span class="type">Processor</span> <span class="variable">processor</span> <span class="operator">=</span> (Processor) wrapper.getCurrentProcessor();</span><br><span class="line"> <span class="keyword">if</span> (getLog().isDebugEnabled()) {</span><br><span class="line"> getLog().debug(sm.getString(<span class="string">"abstractConnectionHandler.connectionsGet"</span>,</span><br><span class="line"> processor, socket));</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 省略代码</span></span><br><span class="line"> <span class="comment">// 直到这里创建 processor</span></span><br><span class="line"> <span class="keyword">if</span> (processor == <span class="literal">null</span>) {</span><br><span class="line"> processor = getProtocol().createProcessor();</span><br><span class="line"> register(processor);</span><br><span class="line"> <span class="keyword">if</span> (getLog().isDebugEnabled()) {</span><br><span class="line"> getLog().debug(sm.getString(<span class="string">"abstractConnectionHandler.processorCreate"</span>, processor));</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>也就是 <code>org.apache.coyote.http11.AbstractHttp11Protocol#createProcessor</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">protected</span> Processor <span class="title function_">createProcessor</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">Http11Processor</span> <span class="variable">processor</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Http11Processor</span>(<span class="built_in">this</span>, adapter);</span><br><span class="line"> <span class="keyword">return</span> processor;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>再往后就是调用 process 方法了，然后它是 Http11Processor 的抽象父类<br><code>org.apache.coyote.AbstractProcessorLight</code><br>会调用 <code>org.apache.coyote.AbstractProcessorLight#process</code> 来处理前面说的 socket<br>接着会跑到这</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> SocketState <span class="title function_">process</span><span class="params">(SocketWrapperBase<?> socketWrapper, SocketEvent status)</span></span><br><span class="line"> <span class="keyword">throws</span> IOException {</span><br><span class="line"></span><br><span class="line"> <span class="type">SocketState</span> <span class="variable">state</span> <span class="operator">=</span> SocketState.CLOSED;</span><br><span class="line"> Iterator<DispatchType> dispatches = <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">do</span> {</span><br><span class="line"> <span class="keyword">if</span> (dispatches != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">DispatchType</span> <span class="variable">nextDispatch</span> <span class="operator">=</span> dispatches.next();</span><br><span class="line"> <span class="keyword">if</span> (getLog().isDebugEnabled()) {</span><br><span class="line"> getLog().debug(<span class="string">"Processing dispatch type: ["</span> + nextDispatch + <span class="string">"]"</span>);</span><br><span class="line"> }</span><br><span class="line"> state = dispatch(nextDispatch.getSocketStatus());</span><br><span class="line"> <span class="keyword">if</span> (!dispatches.hasNext()) {</span><br><span class="line"> state = checkForPipelinedData(state, socketWrapper);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (status == SocketEvent.DISCONNECT) {</span><br><span class="line"> <span class="comment">// Do nothing here, just wait for it to get recycled</span></span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (isAsync() || isUpgrade() || state == SocketState.ASYNC_END) {</span><br><span class="line"> state = dispatch(status);</span><br><span class="line"> state = checkForPipelinedData(state, socketWrapper);</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (status == SocketEvent.OPEN_WRITE) {</span><br><span class="line"> <span class="comment">// Extra write event likely after async, ignore</span></span><br><span class="line"> state = SocketState.LONG;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (status == SocketEvent.OPEN_READ) {</span><br><span class="line"> state = service(socketWrapper);</span><br></pre></td></tr></table></figure>
|
|
<p>接下去就是<br><code>org.apache.coyote.http11.Http11Processor#service</code><br>再就是调用coyote的 service 方法也就是 <code>org.apache.catalina.connector.CoyoteAdapter#service</code><br>这里就会调用到</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"></span><br><span class="line">connector.getService().getContainer().getPipeline().getFirst().invoke(</span><br><span class="line"> request, response);</span><br><span class="line"> </span><br></pre></td></tr></table></figure>
|
|
<p>然后进行 valve 串的执行到 <code>org.apache.catalina.core.StandardWrapperValve#invoke</code> 中<br>会调用</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">filterChain.doFilter</span><br><span class="line"> (request.getRequest(), response.getResponse());</span><br></pre></td></tr></table></figure>
|
|
<p>就会执行 filter 链<br>最后到<br><code>org.apache.catalina.core.ApplicationFilterChain#internalDoFilter</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">servlet.service(request, response);</span><br></pre></td></tr></table></figure>
|
|
<p>就到了 DispatcherServlet 处理的流程, 这样就和之前介绍 DispatcherServlet开始的请求处理接上了。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
<category>Tomcat</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Tomcat</tag>
|
|
<tag>SpringBoot</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Tomcat 系列篇九-介绍下 Tomcat 里的 ContainerBase</title>
|
|
<url>/2023/11/19/Tomcat-%E7%B3%BB%E5%88%97%E7%AF%87%E4%B9%9D-%E4%BB%8B%E7%BB%8D%E4%B8%8B-Tomcat-%E9%87%8C%E7%9A%84-ContainerBase/</url>
|
|
<content><![CDATA[<p>前面介绍过 Tomcat 的层次结构，</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">Server</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">Service</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">Connector</span> /></span></span><br><span class="line"> <span class="tag"><<span class="name">Connector</span> /></span></span><br><span class="line"> <span class="tag"><<span class="name">Engine</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">Host</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">Context</span> /></span></span><br><span class="line"> <span class="tag"></<span class="name">Host</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">Engine</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">Service</span>></span></span><br><span class="line"><span class="tag"></<span class="name">Server</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>参考这个 xml，而对于这些组件中，有一类有相同的基类，也就是这次要介绍的 ContainerBase，<br><img data-src="https://img.nicksxs.me/blog/Adqfbb.png"><br>包括 engine，host，context 还有 wrapper ，都是同样的容器组件，<br>而他们共同实现的接口就是 Container ，<br>主要包含了 </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">ADD_CHILD_EVENT</span> <span class="operator">=</span> <span class="string">"addChild"</span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * The ContainerEvent event type sent when a valve is added</span></span><br><span class="line"><span class="comment"> * by <code>addValve()</code>, if this Container supports pipelines.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">ADD_VALVE_EVENT</span> <span class="operator">=</span> <span class="string">"addValve"</span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * The ContainerEvent event type sent when a child container is removed</span></span><br><span class="line"><span class="comment"> * by <code>removeChild()</code>.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">REMOVE_CHILD_EVENT</span> <span class="operator">=</span> <span class="string">"removeChild"</span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * The ContainerEvent event type sent when a valve is removed</span></span><br><span class="line"><span class="comment"> * by <code>removeValve()</code>, if this Container supports pipelines.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">REMOVE_VALVE_EVENT</span> <span class="operator">=</span> <span class="string">"removeValve"</span>;</span><br></pre></td></tr></table></figure>
|
|
<p>这几个事件类型，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Pipeline <span class="title function_">getPipeline</span><span class="params">()</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> Container <span class="title function_">getParent</span><span class="params">()</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">addChild</span><span class="params">(Container child)</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> Container[] findChildren();</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="type">int</span> <span class="title function_">getStartStopThreads</span><span class="params">()</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">setStartStopThreads</span><span class="params">(<span class="type">int</span> startStopThreads)</span>;</span><br><span class="line"></span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
<p>包括 pipeline 和获取 parent，添加 child 跟查找 child，然后设置线程池的线程数等<br>而对于 ContainerBase<br>也包含了 Lifecycle 的常规方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">initInternal</span><span class="params">()</span> <span class="keyword">throws</span> LifecycleException {</span><br><span class="line"> reconfigureStartStopExecutor(getStartStopThreads());</span><br><span class="line"> <span class="built_in">super</span>.initInternal();</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">reconfigureStartStopExecutor</span><span class="params">(<span class="type">int</span> threads)</span> {</span><br><span class="line"> <span class="keyword">if</span> (threads == <span class="number">1</span>) {</span><br><span class="line"> <span class="comment">// Use a fake executor</span></span><br><span class="line"> <span class="keyword">if</span> (!(startStopExecutor <span class="keyword">instanceof</span> InlineExecutorService)) {</span><br><span class="line"> startStopExecutor = <span class="keyword">new</span> <span class="title class_">InlineExecutorService</span>();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// Delegate utility execution to the Service</span></span><br><span class="line"> <span class="type">Server</span> <span class="variable">server</span> <span class="operator">=</span> Container.getService(<span class="built_in">this</span>).getServer();</span><br><span class="line"> server.setUtilityThreads(threads);</span><br><span class="line"> startStopExecutor = server.getUtilityExecutor();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>初始化方法主要就是之前讲过的线程池<br>后面就开始 startInternal</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title function_">startInternal</span><span class="params">()</span> <span class="keyword">throws</span> LifecycleException {</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Start our subordinate components, if any</span></span><br><span class="line"> logger = <span class="literal">null</span>;</span><br><span class="line"> getLogger();</span><br><span class="line"> <span class="type">Cluster</span> <span class="variable">cluster</span> <span class="operator">=</span> getClusterInternal();</span><br><span class="line"> <span class="keyword">if</span> (cluster <span class="keyword">instanceof</span> Lifecycle) {</span><br><span class="line"> ((Lifecycle) cluster).start();</span><br><span class="line"> }</span><br><span class="line"> <span class="type">Realm</span> <span class="variable">realm</span> <span class="operator">=</span> getRealmInternal();</span><br><span class="line"> <span class="keyword">if</span> (realm <span class="keyword">instanceof</span> Lifecycle) {</span><br><span class="line"> ((Lifecycle) realm).start();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Start our child containers, if any</span></span><br><span class="line"> Container children[] = findChildren();</span><br><span class="line"> List<Future<Void>> results = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"> <span class="keyword">for</span> (Container child : children) {</span><br><span class="line"> results.add(startStopExecutor.submit(<span class="keyword">new</span> <span class="title class_">StartChild</span>(child)));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">MultiThrowable</span> <span class="variable">multiThrowable</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (Future<Void> result : results) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> result.get();</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> log.error(sm.getString(<span class="string">"containerBase.threadedStartFailed"</span>), e);</span><br><span class="line"> <span class="keyword">if</span> (multiThrowable == <span class="literal">null</span>) {</span><br><span class="line"> multiThrowable = <span class="keyword">new</span> <span class="title class_">MultiThrowable</span>();</span><br><span class="line"> }</span><br><span class="line"> multiThrowable.add(e);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (multiThrowable != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">LifecycleException</span>(sm.getString(<span class="string">"containerBase.threadedStartFailed"</span>),</span><br><span class="line"> multiThrowable.getThrowable());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Start the Valves in our pipeline (including the basic), if any</span></span><br><span class="line"> <span class="keyword">if</span> (pipeline <span class="keyword">instanceof</span> Lifecycle) {</span><br><span class="line"> ((Lifecycle) pipeline).start();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> setState(LifecycleState.STARTING);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Start our thread</span></span><br><span class="line"> <span class="keyword">if</span> (backgroundProcessorDelay > <span class="number">0</span>) {</span><br><span class="line"> monitorFuture = Container.getService(ContainerBase.<span class="built_in">this</span>).getServer()</span><br><span class="line"> .getUtilityExecutor().scheduleWithFixedDelay(</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">ContainerBackgroundProcessorMonitor</span>(), <span class="number">0</span>, <span class="number">60</span>, TimeUnit.SECONDS);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里包括了 cluster 的启动和 realm 的启动，然后将 child 子组件添加到线程池里启动，然后循环 get 结果，接下去就是 pipeline 的启动，设置开始中状态，后面是 ContainerBackgroundProcessor 的启动<br>然后是停止方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title function_">stopInternal</span><span class="params">()</span> <span class="keyword">throws</span> LifecycleException {</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Stop our thread</span></span><br><span class="line"> <span class="keyword">if</span> (monitorFuture != <span class="literal">null</span>) {</span><br><span class="line"> monitorFuture.cancel(<span class="literal">true</span>);</span><br><span class="line"> monitorFuture = <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"> threadStop();</span><br><span class="line"></span><br><span class="line"> setState(LifecycleState.STOPPING);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Stop the Valves in our pipeline (including the basic), if any</span></span><br><span class="line"> <span class="keyword">if</span> (pipeline <span class="keyword">instanceof</span> Lifecycle &&</span><br><span class="line"> ((Lifecycle) pipeline).getState().isAvailable()) {</span><br><span class="line"> ((Lifecycle) pipeline).stop();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Stop our child containers, if any</span></span><br><span class="line"> Container children[] = findChildren();</span><br><span class="line"> List<Future<Void>> results = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"> <span class="keyword">for</span> (Container child : children) {</span><br><span class="line"> results.add(startStopExecutor.submit(<span class="keyword">new</span> <span class="title class_">StopChild</span>(child)));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">boolean</span> <span class="variable">fail</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="keyword">for</span> (Future<Void> result : results) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> result.get();</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> log.error(sm.getString(<span class="string">"containerBase.threadedStopFailed"</span>), e);</span><br><span class="line"> fail = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (fail) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">LifecycleException</span>(</span><br><span class="line"> sm.getString(<span class="string">"containerBase.threadedStopFailed"</span>));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Stop our subordinate components, if any</span></span><br><span class="line"> <span class="type">Realm</span> <span class="variable">realm</span> <span class="operator">=</span> getRealmInternal();</span><br><span class="line"> <span class="keyword">if</span> (realm <span class="keyword">instanceof</span> Lifecycle) {</span><br><span class="line"> ((Lifecycle) realm).stop();</span><br><span class="line"> }</span><br><span class="line"> <span class="type">Cluster</span> <span class="variable">cluster</span> <span class="operator">=</span> getClusterInternal();</span><br><span class="line"> <span class="keyword">if</span> (cluster <span class="keyword">instanceof</span> Lifecycle) {</span><br><span class="line"> ((Lifecycle) cluster).stop();</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>也是类似的，只不过停止顺序和刚才开始的顺序反了一下</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
<category>Tomcat</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Tomcat</tag>
|
|
<tag>SpringBoot</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Tomcat 系列篇八-介绍下 Tomcat 里的线程池用处</title>
|
|
<url>/2023/11/05/Tomcat-%E7%B3%BB%E5%88%97%E7%AF%87%E5%85%AB-%E4%BB%8B%E7%BB%8D%E4%B8%8B-Tomcat-%E9%87%8C%E7%9A%84%E7%BA%BF%E7%A8%8B%E6%B1%A0%E7%94%A8%E5%A4%84/</url>
|
|
<content><![CDATA[<p>线程池在 Tomcat 中也是非常重要的工具，这里我们简单介绍下 Tomcat 中的线程池，在 container 的启动过程中<br>org.apache.catalina.core.ContainerBase#initInternal</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">initInternal</span><span class="params">()</span> <span class="keyword">throws</span> LifecycleException {</span><br><span class="line"> reconfigureStartStopExecutor(getStartStopThreads());</span><br><span class="line"> <span class="built_in">super</span>.initInternal();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里先会获取线程数，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="type">int</span> <span class="title function_">getStartStopThreads</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> startStopThreads;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>默认的 ContainerBase 的设置线程数是 1</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="type">int</span> <span class="variable">startStopThreads</span> <span class="operator">=</span> <span class="number">1</span>;</span><br></pre></td></tr></table></figure>
|
|
<p>那就会按照 org.apache.catalina.core.ContainerBase#reconfigureStartStopExecutor 来设置线程池类型</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">reconfigureStartStopExecutor</span><span class="params">(<span class="type">int</span> threads)</span> {</span><br><span class="line"> <span class="keyword">if</span> (threads == <span class="number">1</span>) {</span><br><span class="line"> <span class="comment">// Use a fake executor</span></span><br><span class="line"> <span class="keyword">if</span> (!(startStopExecutor <span class="keyword">instanceof</span> InlineExecutorService)) {</span><br><span class="line"> startStopExecutor = <span class="keyword">new</span> <span class="title class_">InlineExecutorService</span>();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// Delegate utility execution to the Service</span></span><br><span class="line"> <span class="type">Server</span> <span class="variable">server</span> <span class="operator">=</span> Container.getService(<span class="built_in">this</span>).getServer();</span><br><span class="line"> server.setUtilityThreads(threads);</span><br><span class="line"> startStopExecutor = server.getUtilityExecutor();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>此时线程池类型是 null 也就会走到上一个分支中，new 一个 InlineExecutorService 出来<br>前面的这些其实是在 StandardEngine 初始化过程中，也就是 initInternal 时候进行的<br>然后具体的使用是在 startInternal 开始方法中，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// Start our child containers, if any</span></span><br><span class="line">Container children[] = findChildren();</span><br><span class="line">List<Future<Void>> results = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"><span class="keyword">for</span> (Container child : children) {</span><br><span class="line"> results.add(startStopExecutor.submit(<span class="keyword">new</span> <span class="title class_">StartChild</span>(child)));</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>会获取当前 StandardEngine 的子组件提交给线程池进行启动<br>而这个 StartChild 也比较简单</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">class</span> <span class="title class_">StartChild</span> <span class="keyword">implements</span> <span class="title class_">Callable</span><Void> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> Container child;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">StartChild</span><span class="params">(Container child)</span> {</span><br><span class="line"> <span class="built_in">this</span>.child = child;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> Void <span class="title function_">call</span><span class="params">()</span> <span class="keyword">throws</span> LifecycleException {</span><br><span class="line"> child.start();</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>就是调用子组件的 start 方法，<br>在 org.apache.tomcat.util.threads.InlineExecutorService 的父类，java.util.concurrent.AbstractExecutorService 中先会把 Callable 包装成 FutureTask</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <T> Future<T> <span class="title function_">submit</span><span class="params">(Callable<T> task)</span> {</span><br><span class="line"> <span class="keyword">if</span> (task == <span class="literal">null</span>) <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">NullPointerException</span>();</span><br><span class="line"> RunnableFuture<T> ftask = newTaskFor(task);</span><br><span class="line"> execute(ftask);</span><br><span class="line"> <span class="keyword">return</span> ftask;</span><br><span class="line">}</span><br><span class="line"><span class="keyword">protected</span> <T> RunnableFuture<T> <span class="title function_">newTaskFor</span><span class="params">(Callable<T> callable)</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">FutureTask</span><T>(callable);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后进行执行，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">execute</span><span class="params">(Runnable command)</span> {</span><br><span class="line"> <span class="keyword">synchronized</span> (lock) {</span><br><span class="line"> <span class="keyword">if</span> (shutdown) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">RejectedExecutionException</span>();</span><br><span class="line"> }</span><br><span class="line"> taskRunning = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> command.run();</span><br><span class="line"> <span class="keyword">synchronized</span> (lock) {</span><br><span class="line"> taskRunning = <span class="literal">false</span>;</span><br><span class="line"> <span class="keyword">if</span> (shutdown) {</span><br><span class="line"> terminated = <span class="literal">true</span>;</span><br><span class="line"> lock.notifyAll();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这个就是我们 startStopExecutor 的主要作用，帮我们启动子组件<br>对于 Container 来说启动的就是 host，而其实 host 也是继承了 ContainerBase 的，后面可以再继续介绍下</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
<category>Tomcat</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Tomcat</tag>
|
|
<tag>SpringBoot</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Tomcat 系列篇二-介绍下整体架构</title>
|
|
<url>/2023/09/16/Tomcat-%E7%B3%BB%E5%88%97%E7%AF%87%E4%BA%8C-%E4%BB%8B%E7%BB%8D%E4%B8%8B-Engine/</url>
|
|
<content><![CDATA[<p>前面那一篇感觉上来的有点突兀，还是应该按照架构去慢慢解析，所以这里回归下我们整体的 Tomcat 架构，这里我们通过一个 Tomcat 的配置文件来看看</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">Server</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">Service</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">Connector</span> /></span></span><br><span class="line"> <span class="tag"><<span class="name">Connector</span> /></span></span><br><span class="line"> <span class="tag"><<span class="name">Engine</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">Host</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">Context</span> /></span></span><br><span class="line"> <span class="tag"></<span class="name">Host</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">Engine</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">Service</span>></span></span><br><span class="line"><span class="tag"></<span class="name">Server</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>上次我们讲解了 connector，也提到了初始化的流程，在<br><code>org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory#getWebServer</code><br>代码中我们就能略窥一斑，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">Tomcat</span> <span class="variable">tomcat</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Tomcat</span>();</span><br><span class="line"> <span class="type">File</span> <span class="variable">baseDir</span> <span class="operator">=</span> <span class="built_in">this</span>.baseDirectory != <span class="literal">null</span> ? <span class="built_in">this</span>.baseDirectory : <span class="built_in">this</span>.createTempDir(<span class="string">"tomcat"</span>);</span><br><span class="line"> tomcat.setBaseDir(baseDir.getAbsolutePath());</span><br><span class="line"> <span class="type">Connector</span> <span class="variable">connector</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Connector</span>(<span class="built_in">this</span>.protocol);</span><br><span class="line"> connector.setThrowOnFailure(<span class="literal">true</span>);</span><br><span class="line"> tomcat.getService().addConnector(connector);</span><br><span class="line"> <span class="built_in">this</span>.customizeConnector(connector);</span><br><span class="line"> tomcat.setConnector(connector);</span><br><span class="line"> tomcat.getHost().setAutoDeploy(<span class="literal">false</span>);</span><br><span class="line"> <span class="built_in">this</span>.configureEngine(tomcat.getEngine());</span><br></pre></td></tr></table></figure>
|
|
<p>这里的 connector 是在 service 中的，而<br><code>tomcat.getService().addConnector(connector);</code> 这一行<br>具体来看下</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Service <span class="title function_">getService</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> getServer().findServices()[<span class="number">0</span>];</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>又调用了，<code>org.apache.catalina.startup.Tomcat#getServer</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Server <span class="title function_">getServer</span><span class="params">()</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (server != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> server;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> System.setProperty(<span class="string">"catalina.useNaming"</span>, <span class="string">"false"</span>);</span><br><span class="line"></span><br><span class="line"> server = <span class="keyword">new</span> <span class="title class_">StandardServer</span>();</span><br><span class="line"></span><br><span class="line"> initBaseDir();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Set configuration source</span></span><br><span class="line"> ConfigFileLoader.setSource(<span class="keyword">new</span> <span class="title class_">CatalinaBaseConfigurationSource</span>(<span class="keyword">new</span> <span class="title class_">File</span>(basedir), <span class="literal">null</span>));</span><br><span class="line"></span><br><span class="line"> server.setPort( -<span class="number">1</span> );</span><br><span class="line"></span><br><span class="line"> <span class="type">Service</span> <span class="variable">service</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">StandardService</span>();</span><br><span class="line"> service.setName(<span class="string">"Tomcat"</span>);</span><br><span class="line"> server.addService(service);</span><br><span class="line"> <span class="keyword">return</span> server;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>可以看到，先 new 了 StandardServer，再 new 了 StandardService，可以理解为创建 Server 后具体是由 service 进行服务，<br>而在 service 中就是上面的配置文件里显示的，service 包含了 connector，可以是多个connector，负责接入，具体内容可以参看<a href="https://nicksxs.me/2023/09/10/Tomcat-%E7%B3%BB%E5%88%97%E7%AF%87%E4%B8%80-%E4%BB%8B%E7%BB%8D%E4%B8%8B-connector/">上一篇</a><br>而后是 Engine，Engine 的关系是一个 Service 有一个 Engine，Engine 负责处理真正的逻辑，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Engine <span class="title function_">getEngine</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">Service</span> <span class="variable">service</span> <span class="operator">=</span> getServer().findServices()[<span class="number">0</span>];</span><br><span class="line"> <span class="keyword">if</span> (service.getContainer() != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> service.getContainer();</span><br><span class="line"> }</span><br><span class="line"> <span class="type">Engine</span> <span class="variable">engine</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">StandardEngine</span>();</span><br><span class="line"> engine.setName( <span class="string">"Tomcat"</span> );</span><br><span class="line"> engine.setDefaultHost(hostname);</span><br><span class="line"> engine.setRealm(createDefaultRealm());</span><br><span class="line"> service.setContainer(engine);</span><br><span class="line"> <span class="keyword">return</span> engine;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>Engine 一般默认我们初始化的都是 StandardEngine，包括前面的 StandardServer 和StandardService，<br>而对于 host 来说，Engine 中可以包含多个 host，也就是可以处理多个虚拟主机的业务逻辑，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Host <span class="title function_">getHost</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">Engine</span> <span class="variable">engine</span> <span class="operator">=</span> getEngine();</span><br><span class="line"> <span class="keyword">if</span> (engine.findChildren().length > <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">return</span> (Host) engine.findChildren()[<span class="number">0</span>];</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">Host</span> <span class="variable">host</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">StandardHost</span>();</span><br><span class="line"> host.setName(hostname);</span><br><span class="line"> getEngine().addChild(host);</span><br><span class="line"> <span class="keyword">return</span> host;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>tomcat的特点也都是常规的懒加载，在 get 的第一次请求里进行初始化，这边同样创建了 StandardHost，对于可以有多个的 host，在 Engine 中添加也变成了addChild，而对于常规的 Tomcat 来说，往下一层就是 context 了，这个可以支持多个 web 应用，所以也是可以添加多个 context，但我这边以 springboot 嵌入的 Tomcat 举例，他是内嵌的 context</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">prepareContext</span><span class="params">(Host host, ServletContextInitializer[] initializers)</span> {</span><br><span class="line"> <span class="type">File</span> <span class="variable">documentRoot</span> <span class="operator">=</span> <span class="built_in">this</span>.getValidDocumentRoot();</span><br><span class="line"> <span class="type">TomcatEmbeddedContext</span> <span class="variable">context</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">TomcatEmbeddedContext</span>();</span><br><span class="line"> <span class="keyword">if</span> (documentRoot != <span class="literal">null</span>) {</span><br><span class="line"> context.setResources(<span class="keyword">new</span> <span class="title class_">LoaderHidingResourceRoot</span>(context));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> context.setName(<span class="built_in">this</span>.getContextPath());</span><br><span class="line"> context.setDisplayName(<span class="built_in">this</span>.getDisplayName());</span><br><span class="line"> context.setPath(<span class="built_in">this</span>.getContextPath());</span><br><span class="line"> <span class="type">File</span> <span class="variable">docBase</span> <span class="operator">=</span> documentRoot != <span class="literal">null</span> ? documentRoot : <span class="built_in">this</span>.createTempDir(<span class="string">"tomcat-docbase"</span>);</span><br><span class="line"> context.setDocBase(docBase.getAbsolutePath());</span><br><span class="line"> context.addLifecycleListener(<span class="keyword">new</span> <span class="title class_">Tomcat</span>.FixContextListener());</span><br><span class="line"> context.setParentClassLoader(<span class="built_in">this</span>.resourceLoader != <span class="literal">null</span> ? <span class="built_in">this</span>.resourceLoader.getClassLoader() : ClassUtils.getDefaultClassLoader());</span><br><span class="line"> <span class="built_in">this</span>.resetDefaultLocaleMapping(context);</span><br><span class="line"> <span class="built_in">this</span>.addLocaleMappings(context);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> context.setCreateUploadTargets(<span class="literal">true</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (NoSuchMethodError var8) {</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.configureTldPatterns(context);</span><br><span class="line"> <span class="type">WebappLoader</span> <span class="variable">loader</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">WebappLoader</span>();</span><br><span class="line"> loader.setLoaderClass(TomcatEmbeddedWebappClassLoader.class.getName());</span><br><span class="line"> loader.setDelegate(<span class="literal">true</span>);</span><br><span class="line"> context.setLoader(loader);</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.isRegisterDefaultServlet()) {</span><br><span class="line"> <span class="built_in">this</span>.addDefaultServlet(context);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.shouldRegisterJspServlet()) {</span><br><span class="line"> <span class="built_in">this</span>.addJspServlet(context);</span><br><span class="line"> <span class="built_in">this</span>.addJasperInitializer(context);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> context.addLifecycleListener(<span class="keyword">new</span> <span class="title class_">StaticResourceConfigurer</span>(context));</span><br><span class="line"> ServletContextInitializer[] initializersToUse = <span class="built_in">this</span>.mergeInitializers(initializers);</span><br><span class="line"> host.addChild(context);</span><br><span class="line"> <span class="built_in">this</span>.configureContext(context, initializersToUse);</span><br><span class="line"> <span class="built_in">this</span>.postProcessContext(context);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>是一个 <code>TomcatEmbeddedContext</code>，这一点比较特殊，希望这样会有个一个大致的概念。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
<category>Tomcat</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Tomcat</tag>
|
|
<tag>SpringBoot</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Tomcat 系列篇六-介绍下 Lifecycle</title>
|
|
<url>/2023/10/21/Tomcat-%E7%B3%BB%E5%88%97%E7%AF%87%E5%85%AD-%E4%BB%8B%E7%BB%8D%E4%B8%8B-Lifecycle/</url>
|
|
<content><![CDATA[<p>Tomcat 中的很多组件都是继承了LifecycleBase这个抽象类的，包括之前讲过的 connector，server，service，context，host 这些组件都是，我们先来看下接口<br>主体是分为两部分，第一部分是定义状态时间，如 before_init 和 after_init，跟事件监听器的几个方法，添加查询和移除，第二部分是生命周期的相关方法，初始化开始结束与销毁等方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">interface</span> <span class="title class_">Lifecycle</span> {</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> </span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">BEFORE_INIT_EVENT</span> <span class="operator">=</span> <span class="string">"before_init"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">AFTER_INIT_EVENT</span> <span class="operator">=</span> <span class="string">"after_init"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">START_EVENT</span> <span class="operator">=</span> <span class="string">"start"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">BEFORE_START_EVENT</span> <span class="operator">=</span> <span class="string">"before_start"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">AFTER_START_EVENT</span> <span class="operator">=</span> <span class="string">"after_start"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">STOP_EVENT</span> <span class="operator">=</span> <span class="string">"stop"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">BEFORE_STOP_EVENT</span> <span class="operator">=</span> <span class="string">"before_stop"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">AFTER_STOP_EVENT</span> <span class="operator">=</span> <span class="string">"after_stop"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">AFTER_DESTROY_EVENT</span> <span class="operator">=</span> <span class="string">"after_destroy"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">BEFORE_DESTROY_EVENT</span> <span class="operator">=</span> <span class="string">"before_destroy"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">PERIODIC_EVENT</span> <span class="operator">=</span> <span class="string">"periodic"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">CONFIGURE_START_EVENT</span> <span class="operator">=</span> <span class="string">"configure_start"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">CONFIGURE_STOP_EVENT</span> <span class="operator">=</span> <span class="string">"configure_stop"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">addLifecycleListener</span><span class="params">(LifecycleListener listener)</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> LifecycleListener[] findLifecycleListeners();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">removeLifecycleListener</span><span class="params">(LifecycleListener listener)</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">init</span><span class="params">()</span> <span class="keyword">throws</span> LifecycleException;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">start</span><span class="params">()</span> <span class="keyword">throws</span> LifecycleException;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">stop</span><span class="params">()</span> <span class="keyword">throws</span> LifecycleException;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">destroy</span><span class="params">()</span> <span class="keyword">throws</span> LifecycleException;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> LifecycleState <span class="title function_">getState</span><span class="params">()</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> String <span class="title function_">getStateName</span><span class="params">()</span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Marker interface used to indicate that the instance should only be used</span></span><br><span class="line"><span class="comment"> * once. Calling {<span class="doctag">@link</span> #stop()} on an instance that supports this interface</span></span><br><span class="line"><span class="comment"> * will automatically call {<span class="doctag">@link</span> #destroy()} after {<span class="doctag">@link</span> #stop()}</span></span><br><span class="line"><span class="comment"> * completes.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">interface</span> <span class="title class_">SingleUse</span> {</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>对应的就是 Lifecycle 的状态，可以在 LifecycleState 中看到，这个枚举有两个字段组成，available 表示该状态下这个对应的组件状态是否可用，另一个就是对应的事件，比如 INITIALIZING 这个 state 对应的 BEFORE_INIT_EVENT</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">enum</span> <span class="title class_">LifecycleState</span> {</span><br><span class="line"> NEW(<span class="literal">false</span>, <span class="literal">null</span>),</span><br><span class="line"> INITIALIZING(<span class="literal">false</span>, Lifecycle.BEFORE_INIT_EVENT),</span><br><span class="line"> INITIALIZED(<span class="literal">false</span>, Lifecycle.AFTER_INIT_EVENT),</span><br><span class="line"> STARTING_PREP(<span class="literal">false</span>, Lifecycle.BEFORE_START_EVENT),</span><br><span class="line"> STARTING(<span class="literal">true</span>, Lifecycle.START_EVENT),</span><br><span class="line"> STARTED(<span class="literal">true</span>, Lifecycle.AFTER_START_EVENT),</span><br><span class="line"> STOPPING_PREP(<span class="literal">true</span>, Lifecycle.BEFORE_STOP_EVENT),</span><br><span class="line"> STOPPING(<span class="literal">false</span>, Lifecycle.STOP_EVENT),</span><br><span class="line"> STOPPED(<span class="literal">false</span>, Lifecycle.AFTER_STOP_EVENT),</span><br><span class="line"> DESTROYING(<span class="literal">false</span>, Lifecycle.BEFORE_DESTROY_EVENT),</span><br><span class="line"> DESTROYED(<span class="literal">false</span>, Lifecycle.AFTER_DESTROY_EVENT),</span><br><span class="line"> FAILED(<span class="literal">false</span>, <span class="literal">null</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="type">boolean</span> available;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> String lifecycleEvent;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="title function_">LifecycleState</span><span class="params">(<span class="type">boolean</span> available, String lifecycleEvent)</span> {</span><br><span class="line"> <span class="built_in">this</span>.available = available;</span><br><span class="line"> <span class="built_in">this</span>.lifecycleEvent = lifecycleEvent;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * May the public methods other than property getters/setters and lifecycle</span></span><br><span class="line"><span class="comment"> * methods be called for a component in this state? It returns</span></span><br><span class="line"><span class="comment"> * <code>true</code> for any component in any of the following states:</span></span><br><span class="line"><span class="comment"> * <ul></span></span><br><span class="line"><span class="comment"> * <li>{<span class="doctag">@link</span> #STARTING}</li></span></span><br><span class="line"><span class="comment"> * <li>{<span class="doctag">@link</span> #STARTED}</li></span></span><br><span class="line"><span class="comment"> * <li>{<span class="doctag">@link</span> #STOPPING_PREP}</li></span></span><br><span class="line"><span class="comment"> * </ul></span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> <code>true</code> if the component is available for use,</span></span><br><span class="line"><span class="comment"> * otherwise <code>false</code></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">isAvailable</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> available;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> String <span class="title function_">getLifecycleEvent</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> lifecycleEvent;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后可以看看 Lifecycle 的状态流转示意，下面是从 Lifecycle 的代码注释里拷出来的</p>
|
|
<blockquote>
|
|
<p>Common interface for component life cycle methods. Catalina components may implement this interface (as well as the appropriate interface(s) for the functionality they support) in order to provide a consistent mechanism to start and stop the component. The valid state transitions for components that support Lifecycle are:</p>
|
|
</blockquote>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"> start()</span><br><span class="line"> -----------------------------</span><br><span class="line"> | |</span><br><span class="line"> | init() |</span><br><span class="line">NEW -»-- INITIALIZING |</span><br><span class="line">| | | | ------------------«-----------------------</span><br><span class="line">| | |auto | | |</span><br><span class="line">| | \|/ start() \|/ \|/ auto auto stop() |</span><br><span class="line">| | INITIALIZED --»-- STARTING_PREP --»- STARTING --»- STARTED --»--- |</span><br><span class="line">| | | | |</span><br><span class="line">| |destroy()| | |</span><br><span class="line">| --»-----«-- ------------------------«-------------------------------- ^</span><br><span class="line">| | | |</span><br><span class="line">| | \|/ auto auto start() |</span><br><span class="line">| | STOPPING_PREP ----»---- STOPPING ------»----- STOPPED -----»-----</span><br><span class="line">| \|/ ^ | ^</span><br><span class="line">| | stop() | | |</span><br><span class="line">| | -------------------------- | |</span><br><span class="line">| | | | |</span><br><span class="line">| | | destroy() destroy() | |</span><br><span class="line">| | FAILED ----»------ DESTROYING ---«----------------- |</span><br><span class="line">| | ^ | |</span><br><span class="line">| | destroy() | |auto |</span><br><span class="line">| --------»----------------- \|/ |</span><br><span class="line">| DESTROYED |</span><br><span class="line">| |</span><br><span class="line">| stop() |</span><br><span class="line">----»-----------------------------»------------------------------</span><br></pre></td></tr></table></figure>
|
|
<p> Any state can transition to FAILED.</p>
|
|
<p> Calling start() while a component is in states STARTING_PREP, STARTING or<br> STARTED has no effect.</p>
|
|
<p> Calling start() while a component is in state NEW will cause init() to be<br> called immediately after the start() method is entered.</p>
|
|
<p> Calling stop() while a component is in states STOPPING_PREP, STOPPING or<br> STOPPED has no effect.</p>
|
|
<p> Calling stop() while a component is in state NEW transitions the component<br> to STOPPED. This is typically encountered when a component fails to start and<br> does not start all its sub-components. When the component is stopped, it will<br> try to stop all sub-components - even those it didn’t start.</p>
|
|
<p> Attempting any other transition will throw LifecycleException.</p>
|
|
<p>The LifecycleEvents fired during state changes are defined in the methods that trigger the changed. No LifecycleEvents are fired if the attempted transition is not valid.<br>好的项目就是会把这样的示意图画得很好，把状态流转都画得很明确，而不一定要什么好看的作图工具，用字符就可以<br>而在 LifecycleBase类中是把更细节的实现了，<br>init方法就是先判断了状态</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">final</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title function_">init</span><span class="params">()</span> <span class="keyword">throws</span> LifecycleException {</span><br><span class="line"> <span class="keyword">if</span> (!state.equals(LifecycleState.NEW)) {</span><br><span class="line"> <span class="comment">// 只有状态是 new 才可以执行初始化 init</span></span><br><span class="line"> invalidTransition(Lifecycle.BEFORE_INIT_EVENT);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> setStateInternal(LifecycleState.INITIALIZING, <span class="literal">null</span>, <span class="literal">false</span>);</span><br><span class="line"> initInternal();</span><br><span class="line"> setStateInternal(LifecycleState.INITIALIZED, <span class="literal">null</span>, <span class="literal">false</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable t) {</span><br><span class="line"> handleSubClassException(t, <span class="string">"lifecycleBase.initFail"</span>, toString());</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>在判断状态后就先设置了初始化中这个状态，这里设置的时候是 check 是 false 的，所以直接到设置状态，并且触发事件</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title function_">setStateInternal</span><span class="params">(LifecycleState state, Object data, <span class="type">boolean</span> check)</span></span><br><span class="line"> <span class="keyword">throws</span> LifecycleException {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(sm.getString(<span class="string">"lifecycleBase.setState"</span>, <span class="built_in">this</span>, state));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (check) {</span><br><span class="line"> <span class="comment">// Must have been triggered by one of the abstract methods (assume</span></span><br><span class="line"> <span class="comment">// code in this class is correct)</span></span><br><span class="line"> <span class="comment">// null is never a valid state</span></span><br><span class="line"> <span class="keyword">if</span> (state == <span class="literal">null</span>) {</span><br><span class="line"> invalidTransition(<span class="string">"null"</span>);</span><br><span class="line"> <span class="comment">// Unreachable code - here to stop eclipse complaining about</span></span><br><span class="line"> <span class="comment">// a possible NPE further down the method</span></span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Any method can transition to failed</span></span><br><span class="line"> <span class="comment">// startInternal() permits STARTING_PREP to STARTING</span></span><br><span class="line"> <span class="comment">// stopInternal() permits STOPPING_PREP to STOPPING and FAILED to</span></span><br><span class="line"> <span class="comment">// STOPPING</span></span><br><span class="line"> <span class="keyword">if</span> (!(state == LifecycleState.FAILED ||</span><br><span class="line"> (<span class="built_in">this</span>.state == LifecycleState.STARTING_PREP &&</span><br><span class="line"> state == LifecycleState.STARTING) ||</span><br><span class="line"> (<span class="built_in">this</span>.state == LifecycleState.STOPPING_PREP &&</span><br><span class="line"> state == LifecycleState.STOPPING) ||</span><br><span class="line"> (<span class="built_in">this</span>.state == LifecycleState.FAILED &&</span><br><span class="line"> state == LifecycleState.STOPPING))) {</span><br><span class="line"> <span class="comment">// No other transition permitted</span></span><br><span class="line"> invalidTransition(state.name());</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.state = state;</span><br><span class="line"> <span class="type">String</span> <span class="variable">lifecycleEvent</span> <span class="operator">=</span> state.getLifecycleEvent();</span><br><span class="line"> <span class="keyword">if</span> (lifecycleEvent != <span class="literal">null</span>) {</span><br><span class="line"> fireLifecycleEvent(lifecycleEvent, data);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>触发事件调用了 fireLifecycleEvent</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">fireLifecycleEvent</span><span class="params">(String type, Object data)</span> {</span><br><span class="line"> <span class="type">LifecycleEvent</span> <span class="variable">event</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">LifecycleEvent</span>(<span class="built_in">this</span>, type, data);</span><br><span class="line"> <span class="keyword">for</span> (LifecycleListener listener : lifecycleListeners) {</span><br><span class="line"> listener.lifecycleEvent(event);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>会调用所有的 listener 将事件发送出去<br>而回到上面 init 方法，第二步的 initInternal 就是 Tomcat 的 Lifecycle 中非常核心的一点了<br>因为上面说到很多组件都是继承了 LifecycleBase 的，实际外部被调用的其实都是 LifecycleBase 的<br> init，start 等方法，内部是由这个抽象基类去设置前后的状态，以及调用 initInternal ，也就是具体<br> 实现类的实现方法，譬如 connector 的 initInternal 方法，就是真正处理初始化逻辑的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">initInternal</span><span class="params">()</span> <span class="keyword">throws</span> LifecycleException {</span><br><span class="line"></span><br><span class="line"> <span class="built_in">super</span>.initInternal();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (protocolHandler == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">LifecycleException</span>(</span><br><span class="line"> sm.getString(<span class="string">"coyoteConnector.protocolHandlerInstantiationFailed"</span>));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Initialize adapter</span></span><br><span class="line"> adapter = <span class="keyword">new</span> <span class="title class_">CoyoteAdapter</span>(<span class="built_in">this</span>);</span><br><span class="line"> protocolHandler.setAdapter(adapter);</span><br><span class="line"> <span class="keyword">if</span> (service != <span class="literal">null</span>) {</span><br><span class="line"> protocolHandler.setUtilityExecutor(service.getServer().getUtilityExecutor());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Make sure parseBodyMethodsSet has a default</span></span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == parseBodyMethodsSet) {</span><br><span class="line"> setParseBodyMethods(getParseBodyMethods());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (protocolHandler.isAprRequired() && !AprStatus.isInstanceCreated()) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">LifecycleException</span>(sm.getString(<span class="string">"coyoteConnector.protocolHandlerNoAprListener"</span>,</span><br><span class="line"> getProtocolHandlerClassName()));</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (protocolHandler.isAprRequired() && !AprStatus.isAprAvailable()) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">LifecycleException</span>(sm.getString(<span class="string">"coyoteConnector.protocolHandlerNoAprLibrary"</span>,</span><br><span class="line"> getProtocolHandlerClassName()));</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (AprStatus.isAprAvailable() && AprStatus.getUseOpenSSL() &&</span><br><span class="line"> protocolHandler <span class="keyword">instanceof</span> AbstractHttp11JsseProtocol) {</span><br><span class="line"> AbstractHttp11JsseProtocol<?> jsseProtocolHandler =</span><br><span class="line"> (AbstractHttp11JsseProtocol<?>) protocolHandler;</span><br><span class="line"> <span class="keyword">if</span> (jsseProtocolHandler.isSSLEnabled() &&</span><br><span class="line"> jsseProtocolHandler.getSslImplementationName() == <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// OpenSSL is compatible with the JSSE configuration, so use it if APR is available</span></span><br><span class="line"> jsseProtocolHandler.setSslImplementationName(OpenSSLImplementation.class.getName());</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> protocolHandler.init();</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">LifecycleException</span>(</span><br><span class="line"> sm.getString(<span class="string">"coyoteConnector.protocolHandlerInitializationFailed"</span>), e);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>start 方法也是类似的，但是更复杂，第一步也是状态判断，是不是已经被启动过了，如果还是 new 状态的话就先调用 init，如果已经 failed 启动失败了就直接调用 stop 结束了，如果是非已初始化和非已停止的状态则是错误的状态转变<br>如果是正常情况，那就是先设置成 STARTING_PREP，然后调用实现类的 startInternal，如果启动失败了就调用 stop，此时理论上应该是启动中 STARTING 状态，否则就是异常状态转变，剩下的就是状态启动完的状态，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">final</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title function_">start</span><span class="params">()</span> <span class="keyword">throws</span> LifecycleException {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (LifecycleState.STARTING_PREP.equals(state) || LifecycleState.STARTING.equals(state) ||</span><br><span class="line"> LifecycleState.STARTED.equals(state)) {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> <span class="type">Exception</span> <span class="variable">e</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">LifecycleException</span>();</span><br><span class="line"> log.debug(sm.getString(<span class="string">"lifecycleBase.alreadyStarted"</span>, toString()), e);</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (log.isInfoEnabled()) {</span><br><span class="line"> log.info(sm.getString(<span class="string">"lifecycleBase.alreadyStarted"</span>, toString()));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (state.equals(LifecycleState.NEW)) {</span><br><span class="line"> init();</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (state.equals(LifecycleState.FAILED)) {</span><br><span class="line"> stop();</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (!state.equals(LifecycleState.INITIALIZED) &&</span><br><span class="line"> !state.equals(LifecycleState.STOPPED)) {</span><br><span class="line"> invalidTransition(Lifecycle.BEFORE_START_EVENT);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> setStateInternal(LifecycleState.STARTING_PREP, <span class="literal">null</span>, <span class="literal">false</span>);</span><br><span class="line"> startInternal();</span><br><span class="line"> <span class="keyword">if</span> (state.equals(LifecycleState.FAILED)) {</span><br><span class="line"> <span class="comment">// This is a 'controlled' failure. The component put itself into the</span></span><br><span class="line"> <span class="comment">// FAILED state so call stop() to complete the clean-up.</span></span><br><span class="line"> stop();</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (!state.equals(LifecycleState.STARTING)) {</span><br><span class="line"> <span class="comment">// Shouldn't be necessary but acts as a check that sub-classes are</span></span><br><span class="line"> <span class="comment">// doing what they are supposed to.</span></span><br><span class="line"> invalidTransition(Lifecycle.AFTER_START_EVENT);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> setStateInternal(LifecycleState.STARTED, <span class="literal">null</span>, <span class="literal">false</span>);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable t) {</span><br><span class="line"> <span class="comment">// This is an 'uncontrolled' failure so put the component into the</span></span><br><span class="line"> <span class="comment">// FAILED state and throw an exception.</span></span><br><span class="line"> handleSubClassException(t, <span class="string">"lifecycleBase.startFail"</span>, toString());</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>stop 跟 destroy 也类似的</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
<category>Tomcat</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Tomcat</tag>
|
|
<tag>SpringBoot</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Tomcat 系列篇十-介绍下 Tomcat 里的 Mapper 作用</title>
|
|
<url>/2023/11/26/Tomcat-%E7%B3%BB%E5%88%97%E7%AF%87%E5%8D%81-%E4%BB%8B%E7%BB%8D%E4%B8%8B-Tomcat-%E9%87%8C%E7%9A%84-Mapper-%E4%BD%9C%E7%94%A8/</url>
|
|
<content><![CDATA[<p>Mapper 顾名思义是作一个映射作用，在 Tomcat 中会根据域名找到 host 组件，再根据 uri 可以找到对应的 context 和 wrapper 组件，但是对于当前这个环境 (Springboot) 会有一点小区别<br>之前说到<br>请求会经过 coyote 适配器进行进一步处理，<br><code>org.apache.coyote.http11.Http11Processor#service</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// Process the request in the adapter</span></span><br><span class="line"><span class="keyword">if</span> (getErrorState().isIoAllowed()) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> rp.setStage(org.apache.coyote.Constants.STAGE_SERVICE);</span><br><span class="line"> getAdapter().service(request, response);</span><br></pre></td></tr></table></figure>
|
|
<p>然后到 coyoteAdapter 的 service<br><code>org.apache.catalina.connector.CoyoteAdapter#service</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// Parse and set Catalina and configuration specific</span></span><br><span class="line"> <span class="comment">// request parameters</span></span><br><span class="line"> postParseSuccess = postParseRequest(req, request, res, response);</span><br><span class="line"> <span class="keyword">if</span> (postParseSuccess) {</span><br><span class="line"> <span class="comment">//check valves if we support async</span></span><br><span class="line"> request.setAsyncSupported(</span><br><span class="line"> connector.getService().getContainer().getPipeline().isAsyncSupported());</span><br><span class="line"> <span class="comment">// Calling the container</span></span><br><span class="line"> <span class="comment">// ----------> 到这就是调用 pipeline 去处理了，我们要关注上面的 postParseRequest</span></span><br><span class="line"> connector.getService().getContainer().getPipeline().getFirst().invoke(</span><br><span class="line"> request, response);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>主要先看到 <code>postParseRequest</code><br>在 postParseRequest 的代码里会调用 Mapper 的 map 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">while</span> (mapRequired) {</span><br><span class="line"> <span class="comment">// This will map the the latest version by default</span></span><br><span class="line"> connector.getService().getMapper().map(serverName, decodedURI,</span><br><span class="line"> version, request.getMappingData());</span><br><span class="line"></span><br><span class="line"> <span class="comment">// If there is no context at this point, either this is a 404</span></span><br><span class="line"> <span class="comment">// because no ROOT context has been deployed or the URI was invalid</span></span><br><span class="line"> <span class="comment">// so no context could be mapped.</span></span><br><span class="line"> <span class="keyword">if</span> (request.getContext() == <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// Allow processing to continue.</span></span><br></pre></td></tr></table></figure>
|
|
<p>而后面的 context 就是在 map 方法里处理塞进去的<br>往里看就是 <code>org.apache.catalina.mapper.Mapper#internalMap</code><br>第一步找的是 host</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// Virtual host mapping</span></span><br><span class="line"> MappedHost[] hosts = <span class="built_in">this</span>.hosts;</span><br><span class="line"> <span class="type">MappedHost</span> <span class="variable">mappedHost</span> <span class="operator">=</span> exactFindIgnoreCase(hosts, host);</span><br></pre></td></tr></table></figure>
|
|
<p>而在后续代码里继续设置 context</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">if</span> (contextVersion == <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// Return the latest version</span></span><br><span class="line"> <span class="comment">// The versions array is known to contain at least one element</span></span><br><span class="line"> contextVersion = contextVersions[versionCount - <span class="number">1</span>];</span><br><span class="line"> }</span><br><span class="line"> mappingData.context = contextVersion.object;</span><br></pre></td></tr></table></figure>
|
|
<p>然后是 <code>wrapper</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// Wrapper mapping</span></span><br><span class="line"><span class="keyword">if</span> (!contextVersion.isPaused()) {</span><br><span class="line"> internalMapWrapper(contextVersion, uri, mappingData);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>在这个方法 <code>org.apache.catalina.mapper.Mapper#internalMapWrapper</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// Rule 7 -- Default servlet</span></span><br><span class="line"><span class="keyword">if</span> (mappingData.wrapper == <span class="literal">null</span> && !checkJspWelcomeFiles) {</span><br><span class="line"> <span class="keyword">if</span> (contextVersion.defaultWrapper != <span class="literal">null</span>) {</span><br><span class="line"> mappingData.wrapper = contextVersion.defaultWrapper.object;</span><br><span class="line"> mappingData.requestPath.setChars</span><br><span class="line"> (path.getBuffer(), path.getStart(), path.getLength());</span><br><span class="line"> mappingData.wrapperPath.setChars</span><br><span class="line"> (path.getBuffer(), path.getStart(), path.getLength());</span><br><span class="line"> mappingData.matchType = MappingMatch.DEFAULT;</span><br></pre></td></tr></table></figure>
|
|
<p>这里就设置了 wrapper ，因为我们是在 Springboot 中，所以只有默认的 dispatchServlet<br>上面主要是在请求处理过程中的查找映射过程，一开始的注册是从 MapperListener 开始的<br>MapperListener 继承了 LifecycleMbeanBase，也就是有了 Lifecycle 状态变化那一套</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">startInternal</span><span class="params">()</span> <span class="keyword">throws</span> LifecycleException {</span><br><span class="line"></span><br><span class="line"> setState(LifecycleState.STARTING);</span><br><span class="line"></span><br><span class="line"> <span class="type">Engine</span> <span class="variable">engine</span> <span class="operator">=</span> service.getContainer();</span><br><span class="line"> <span class="keyword">if</span> (engine == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> findDefaultHost();</span><br><span class="line"></span><br><span class="line"> addListeners(engine);</span><br><span class="line"></span><br><span class="line"> Container[] conHosts = engine.findChildren();</span><br><span class="line"> <span class="keyword">for</span> (Container conHost : conHosts) {</span><br><span class="line"> <span class="type">Host</span> <span class="variable">host</span> <span class="operator">=</span> (Host) conHost;</span><br><span class="line"> <span class="keyword">if</span> (!LifecycleState.NEW.equals(host.getState())) {</span><br><span class="line"> <span class="comment">// Registering the host will register the context and wrappers</span></span><br><span class="line"> registerHost(host);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>在启动过程中就会去把 engine 的子容器 host 找出来进行注册，就是调用 registerHost 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">registerHost</span><span class="params">(Host host)</span> {</span><br><span class="line"></span><br><span class="line"> String[] aliases = host.findAliases();</span><br><span class="line"> mapper.addHost(host.getName(), aliases, host);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (Container container : host.findChildren()) {</span><br><span class="line"> <span class="keyword">if</span> (container.getState().isAvailable()) {</span><br><span class="line"> registerContext((Context) container);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Default host may have changed</span></span><br><span class="line"> findDefaultHost();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span>(log.isDebugEnabled()) {</span><br><span class="line"> log.debug(sm.getString(<span class="string">"mapperListener.registerHost"</span>,</span><br><span class="line"> host.getName(), domain, service));</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>在这里面会添加 host 组件，注册 context 等，注册context 里还会处理 wrapper 的添加记录</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">registerContext</span><span class="params">(Context context)</span> {</span><br><span class="line"></span><br><span class="line"> <span class="type">String</span> <span class="variable">contextPath</span> <span class="operator">=</span> context.getPath();</span><br><span class="line"> <span class="keyword">if</span> (<span class="string">"/"</span>.equals(contextPath)) {</span><br><span class="line"> contextPath = <span class="string">""</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">Host</span> <span class="variable">host</span> <span class="operator">=</span> (Host)context.getParent();</span><br><span class="line"></span><br><span class="line"> <span class="type">WebResourceRoot</span> <span class="variable">resources</span> <span class="operator">=</span> context.getResources();</span><br><span class="line"> String[] welcomeFiles = context.findWelcomeFiles();</span><br><span class="line"> List<WrapperMappingInfo> wrappers = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (Container container : context.findChildren()) {</span><br><span class="line"> prepareWrapperMappingInfo(context, (Wrapper) container, wrappers);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span>(log.isDebugEnabled()) {</span><br><span class="line"> log.debug(sm.getString(<span class="string">"mapperListener.registerWrapper"</span>,</span><br><span class="line"> container.getName(), contextPath, service));</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> mapper.addContextVersion(host.getName(), host, contextPath,</span><br><span class="line"> context.getWebappVersion(), context, welcomeFiles, resources,</span><br><span class="line"> wrappers);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span>(log.isDebugEnabled()) {</span><br><span class="line"> log.debug(sm.getString(<span class="string">"mapperListener.registerContext"</span>,</span><br><span class="line"> contextPath, service));</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>就是在 <code>org.apache.catalina.mapper.Mapper#addContextVersion</code> 方法中</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">addContextVersion</span><span class="params">(String hostName, Host host, String path,</span></span><br><span class="line"><span class="params"> String version, Context context, String[] welcomeResources,</span></span><br><span class="line"><span class="params"> WebResourceRoot resources, Collection<WrapperMappingInfo> wrappers)</span> {</span><br><span class="line"></span><br><span class="line"> hostName = renameWildcardHost(hostName);</span><br><span class="line"></span><br><span class="line"> <span class="type">MappedHost</span> <span class="variable">mappedHost</span> <span class="operator">=</span> exactFind(hosts, hostName);</span><br><span class="line"> <span class="keyword">if</span> (mappedHost == <span class="literal">null</span>) {</span><br><span class="line"> addHost(hostName, <span class="keyword">new</span> <span class="title class_">String</span>[<span class="number">0</span>], host);</span><br><span class="line"> mappedHost = exactFind(hosts, hostName);</span><br><span class="line"> <span class="keyword">if</span> (mappedHost == <span class="literal">null</span>) {</span><br><span class="line"> log.error(sm.getString(<span class="string">"mapper.addContext.noHost"</span>, hostName));</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (mappedHost.isAlias()) {</span><br><span class="line"> log.error(sm.getString(<span class="string">"mapper.addContext.hostIsAlias"</span>, hostName));</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">int</span> <span class="variable">slashCount</span> <span class="operator">=</span> slashCount(path);</span><br><span class="line"> <span class="keyword">synchronized</span> (mappedHost) {</span><br><span class="line"> <span class="type">ContextVersion</span> <span class="variable">newContextVersion</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ContextVersion</span>(version,</span><br><span class="line"> path, slashCount, context, resources, welcomeResources);</span><br><span class="line"> <span class="keyword">if</span> (wrappers != <span class="literal">null</span>) {</span><br><span class="line"> addWrappers(newContextVersion, wrappers);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>大致介绍了下 Mapper 的逻辑</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
<category>Tomcat</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Tomcat</tag>
|
|
<tag>SpringBoot</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Tomcat 系列篇十一-介绍下 Tomcat 里的后台处理和热加载</title>
|
|
<url>/2023/12/10/Tomcat-%E7%B3%BB%E5%88%97%E7%AF%87%E5%8D%81%E4%B8%80-%E4%BB%8B%E7%BB%8D%E4%B8%8B-Tomcat-%E9%87%8C%E7%9A%84%E5%90%8E%E5%8F%B0%E5%A4%84%E7%90%86%E5%92%8C%E7%83%AD%E5%8A%A0%E8%BD%BD/</url>
|
|
<content><![CDATA[<p>这部分其实之前在讲线程池的时候也有点带到了， 主要是在这个类里<br><code>org.apache.catalina.core.ContainerBase.ContainerBackgroundProcessor</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">class</span> <span class="title class_">ContainerBackgroundProcessor</span> <span class="keyword">implements</span> <span class="title class_">Runnable</span> {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> processChildren(ContainerBase.<span class="built_in">this</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">processChildren</span><span class="params">(Container container)</span> {</span><br><span class="line"> <span class="type">ClassLoader</span> <span class="variable">originalClassLoader</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (container <span class="keyword">instanceof</span> Context) {</span><br><span class="line"> <span class="type">Loader</span> <span class="variable">loader</span> <span class="operator">=</span> ((Context) container).getLoader();</span><br><span class="line"> <span class="comment">// Loader will be null for FailedContext instances</span></span><br><span class="line"> <span class="keyword">if</span> (loader == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Ensure background processing for Contexts and Wrappers</span></span><br><span class="line"> <span class="comment">// is performed under the web app's class loader</span></span><br><span class="line"> originalClassLoader = ((Context) container).bind(<span class="literal">false</span>, <span class="literal">null</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 调用 Container 的 backgroundProcess</span></span><br><span class="line"> container.backgroundProcess();</span><br><span class="line"> <span class="comment">// 然后寻找 children</span></span><br><span class="line"> Container[] children = container.findChildren();</span><br><span class="line"> <span class="keyword">for</span> (Container child : children) {</span><br><span class="line"> <span class="comment">// 如果 backgroundProcessorDelay <= 0 就调用执行</span></span><br><span class="line"> <span class="comment">// 否则代表这个 Container 有之前第八篇说的 StartChild 这种</span></span><br><span class="line"> <span class="keyword">if</span> (child.getBackgroundProcessorDelay() <= <span class="number">0</span>) {</span><br><span class="line"> processChildren(child);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable t) {</span><br><span class="line"> ExceptionUtils.handleThrowable(t);</span><br><span class="line"> log.error(sm.getString(<span class="string">"containerBase.backgroundProcess.error"</span>), t);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="keyword">if</span> (container <span class="keyword">instanceof</span> Context) {</span><br><span class="line"> ((Context) container).unbind(<span class="literal">false</span>, originalClassLoader);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这个触发方式是在 ContainerBase 里的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">class</span> <span class="title class_">ContainerBackgroundProcessorMonitor</span> <span class="keyword">implements</span> <span class="title class_">Runnable</span> {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (getState().isAvailable()) {</span><br><span class="line"> threadStart();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>而在这个 threadStart 里</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">threadStart</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (backgroundProcessorDelay > <span class="number">0</span></span><br><span class="line"> && (getState().isAvailable() || LifecycleState.STARTING_PREP.equals(getState()))</span><br><span class="line"> && (backgroundProcessorFuture == <span class="literal">null</span> || backgroundProcessorFuture.isDone())) {</span><br><span class="line"> <span class="keyword">if</span> (backgroundProcessorFuture != <span class="literal">null</span> && backgroundProcessorFuture.isDone()) {</span><br><span class="line"> <span class="comment">// There was an error executing the scheduled task, get it and log it</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> backgroundProcessorFuture.get();</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException | ExecutionException e) {</span><br><span class="line"> log.error(sm.getString(<span class="string">"containerBase.backgroundProcess.error"</span>), e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> backgroundProcessorFuture = Container.getService(<span class="built_in">this</span>).getServer().getUtilityExecutor()</span><br><span class="line"> .scheduleWithFixedDelay(<span class="keyword">new</span> <span class="title class_">ContainerBackgroundProcessor</span>(),</span><br><span class="line"> backgroundProcessorDelay, backgroundProcessorDelay,</span><br><span class="line"> TimeUnit.SECONDS);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>就调用了线程池的 <code>scheduleWithFixedDelay</code> 方法提交了这个 <code>ContainerBackgroundProcessor</code>，<br>仔细看代码会发现，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">StandardEngine</span><span class="params">()</span> {</span><br><span class="line"></span><br><span class="line"> <span class="built_in">super</span>();</span><br><span class="line"> pipeline.setBasic(<span class="keyword">new</span> <span class="title class_">StandardEngineValve</span>());</span><br><span class="line"> <span class="comment">/* Set the jmvRoute using the system property jvmRoute */</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> setJvmRoute(System.getProperty(<span class="string">"jvmRoute"</span>));</span><br><span class="line"> } <span class="keyword">catch</span>(Exception ex) {</span><br><span class="line"> log.warn(sm.getString(<span class="string">"standardEngine.jvmRouteFail"</span>));</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// By default, the engine will hold the reloading thread</span></span><br><span class="line"> backgroundProcessorDelay = <span class="number">10</span>;</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这个就不用开启后台热加载，而主要的热加载同学应该是<br><code>org.apache.catalina.core.StandardContext#backgroundProcess</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">backgroundProcess</span><span class="params">()</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!getState().isAvailable()) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">Loader</span> <span class="variable">loader</span> <span class="operator">=</span> getLoader();</span><br><span class="line"> <span class="keyword">if</span> (loader != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 这里就用了 loader 的 backgroundProcess</span></span><br><span class="line"> loader.backgroundProcess();</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> log.warn(sm.getString(</span><br><span class="line"> <span class="string">"standardContext.backgroundProcess.loader"</span>, loader), e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="type">Manager</span> <span class="variable">manager</span> <span class="operator">=</span> getManager();</span><br><span class="line"> <span class="keyword">if</span> (manager != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> manager.backgroundProcess();</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> log.warn(sm.getString(</span><br><span class="line"> <span class="string">"standardContext.backgroundProcess.manager"</span>, manager),</span><br><span class="line"> e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="type">WebResourceRoot</span> <span class="variable">resources</span> <span class="operator">=</span> getResources();</span><br><span class="line"> <span class="keyword">if</span> (resources != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> resources.backgroundProcess();</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> log.warn(sm.getString(</span><br><span class="line"> <span class="string">"standardContext.backgroundProcess.resources"</span>,</span><br><span class="line"> resources), e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="type">InstanceManager</span> <span class="variable">instanceManager</span> <span class="operator">=</span> getInstanceManager();</span><br><span class="line"> <span class="keyword">if</span> (instanceManager != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> instanceManager.backgroundProcess();</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> log.warn(sm.getString(</span><br><span class="line"> <span class="string">"standardContext.backgroundProcess.instanceManager"</span>,</span><br><span class="line"> resources), e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">super</span>.backgroundProcess();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>loader 的后台处理就是</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">backgroundProcess</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (reloadable && modified()) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> Thread.currentThread().setContextClassLoader</span><br><span class="line"> (WebappLoader.class.getClassLoader());</span><br><span class="line"> <span class="keyword">if</span> (context != <span class="literal">null</span>) {</span><br><span class="line"> context.reload();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="keyword">if</span> (context != <span class="literal">null</span> && context.getLoader() != <span class="literal">null</span>) {</span><br><span class="line"> Thread.currentThread().setContextClassLoader</span><br><span class="line"> (context.getLoader().getClassLoader());</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后又会回到 context 的 reload，也就是 StandardContext 的 reload</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title function_">reload</span><span class="params">()</span> {</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Validate our current component state</span></span><br><span class="line"> <span class="keyword">if</span> (!getState().isAvailable()) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalStateException</span></span><br><span class="line"> (sm.getString(<span class="string">"standardContext.notStarted"</span>, getName()));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span>(log.isInfoEnabled()) {</span><br><span class="line"> log.info(sm.getString(<span class="string">"standardContext.reloadingStarted"</span>,</span><br><span class="line"> getName()));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Stop accepting requests temporarily.</span></span><br><span class="line"> setPaused(<span class="literal">true</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> stop();</span><br><span class="line"> } <span class="keyword">catch</span> (LifecycleException e) {</span><br><span class="line"> log.error(</span><br><span class="line"> sm.getString(<span class="string">"standardContext.stoppingContext"</span>, getName()), e);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> start();</span><br><span class="line"> } <span class="keyword">catch</span> (LifecycleException e) {</span><br><span class="line"> log.error(</span><br><span class="line"> sm.getString(<span class="string">"standardContext.startingContext"</span>, getName()), e);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> setPaused(<span class="literal">false</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span>(log.isInfoEnabled()) {</span><br><span class="line"> log.info(sm.getString(<span class="string">"standardContext.reloadingCompleted"</span>,</span><br><span class="line"> getName()));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这样就是线程池结合后台处理，还是有些复杂的。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
<category>Tomcat</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Tomcat</tag>
|
|
<tag>SpringBoot</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Tomcat 系列篇十二-番外介绍下 Tomcat 的上传文件限制</title>
|
|
<url>/2023/12/17/Tomcat-%E7%B3%BB%E5%88%97%E7%AF%87%E5%8D%81%E4%BA%8C-%E7%95%AA%E5%A4%96%E4%BB%8B%E7%BB%8D%E4%B8%8B-Tomcat-%E7%9A%84%E4%B8%8A%E4%BC%A0%E6%96%87%E4%BB%B6%E9%99%90%E5%88%B6/</url>
|
|
<content><![CDATA[<p>最近同学在把 springboot 升级到 2.x 版本的过程中碰到了小问题，可能升级变更里能找到信息，不过我们以学习为目的，可以看看代码是怎么样的<br>报错是在这段代码里的<br><code>org.apache.tomcat.util.http.fileupload.util.LimitedInputStream#checkLimit</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">checkLimit</span><span class="params">()</span> <span class="keyword">throws</span> IOException {</span><br><span class="line"> <span class="keyword">if</span> (count > sizeMax) {</span><br><span class="line"> raiseError(sizeMax, count);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>其中的 <code>raiseError</code> 是个抽象方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">abstract</span> <span class="keyword">void</span> <span class="title function_">raiseError</span><span class="params">(<span class="type">long</span> pSizeMax, <span class="type">long</span> pCount)</span></span><br><span class="line"> <span class="keyword">throws</span> IOException;</span><br></pre></td></tr></table></figure>
|
|
<p>具体的实现是在</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">FileItemStreamImpl</span><span class="params">(FileItemIteratorImpl pFileItemIterator, String pName, String pFieldName, String pContentType, <span class="type">boolean</span> pFormField, <span class="type">long</span> pContentLength)</span> <span class="keyword">throws</span> FileUploadException, IOException {</span><br><span class="line"> <span class="built_in">this</span>.fileItemIteratorImpl = pFileItemIterator;</span><br><span class="line"> <span class="built_in">this</span>.name = pName;</span><br><span class="line"> <span class="built_in">this</span>.fieldName = pFieldName;</span><br><span class="line"> <span class="built_in">this</span>.contentType = pContentType;</span><br><span class="line"> <span class="built_in">this</span>.formField = pFormField;</span><br><span class="line"> <span class="type">long</span> <span class="variable">fileSizeMax</span> <span class="operator">=</span> <span class="built_in">this</span>.fileItemIteratorImpl.getFileSizeMax();</span><br><span class="line"> <span class="keyword">if</span> (fileSizeMax != -<span class="number">1L</span> && pContentLength != -<span class="number">1L</span> && pContentLength > fileSizeMax) {</span><br><span class="line"> <span class="type">FileSizeLimitExceededException</span> <span class="variable">e</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">FileSizeLimitExceededException</span>(String.format(<span class="string">"The field %s exceeds its maximum permitted size of %s bytes."</span>, <span class="built_in">this</span>.fieldName, fileSizeMax), pContentLength, fileSizeMax);</span><br><span class="line"> e.setFileName(pName);</span><br><span class="line"> e.setFieldName(pFieldName);</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">FileUploadIOException</span>(e);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">final</span> MultipartStream.<span class="type">ItemInputStream</span> <span class="variable">itemStream</span> <span class="operator">=</span> <span class="built_in">this</span>.fileItemIteratorImpl.getMultiPartStream().newInputStream();</span><br><span class="line"> <span class="type">InputStream</span> <span class="variable">istream</span> <span class="operator">=</span> itemStream;</span><br><span class="line"> <span class="keyword">if</span> (fileSizeMax != -<span class="number">1L</span>) {</span><br><span class="line"> istream = <span class="keyword">new</span> <span class="title class_">LimitedInputStream</span>(itemStream, fileSizeMax) {</span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">raiseError</span><span class="params">(<span class="type">long</span> pSizeMax, <span class="type">long</span> pCount)</span> <span class="keyword">throws</span> IOException {</span><br><span class="line"> itemStream.close(<span class="literal">true</span>);</span><br><span class="line"> <span class="type">FileSizeLimitExceededException</span> <span class="variable">e</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">FileSizeLimitExceededException</span>(String.format(<span class="string">"The field %s exceeds its maximum permitted size of %s bytes."</span>, FileItemStreamImpl.<span class="built_in">this</span>.fieldName, pSizeMax), pCount, pSizeMax);</span><br><span class="line"> e.setFieldName(FileItemStreamImpl.<span class="built_in">this</span>.fieldName);</span><br><span class="line"> e.setFileName(FileItemStreamImpl.<span class="built_in">this</span>.name);</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">FileUploadIOException</span>(e);</span><br><span class="line"> }</span><br><span class="line"> };</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.stream = (InputStream)istream;</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>后面也会介绍到，这里我们其实主要是要找到这个 pSizeMax 是哪里来的<br>通过阅读代码会发现跟这个类 <code>MultipartConfigElement</code> 有关系<br>而在升级后的 springboot 中这个类已经有了自动装配类，也就是<br><code>org.springframework.boot.autoconfigure.web.servlet.MultipartAutoConfiguration</code></p>
|
|
<p>有了这个自动装配</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Configuration(</span></span><br><span class="line"><span class="meta"> proxyBeanMethods = false</span></span><br><span class="line"><span class="meta">)</span></span><br><span class="line"><span class="meta">@ConditionalOnClass({Servlet.class, StandardServletMultipartResolver.class, MultipartConfigElement.class})</span></span><br><span class="line"><span class="meta">@ConditionalOnProperty(</span></span><br><span class="line"><span class="meta"> prefix = "spring.servlet.multipart",</span></span><br><span class="line"><span class="meta"> name = {"enabled"},</span></span><br><span class="line"><span class="meta"> matchIfMissing = true</span></span><br><span class="line"><span class="meta">)</span></span><br><span class="line"><span class="meta">@ConditionalOnWebApplication(</span></span><br><span class="line"><span class="meta"> type = Type.SERVLET</span></span><br><span class="line"><span class="meta">)</span></span><br><span class="line"><span class="meta">@EnableConfigurationProperties({MultipartProperties.class})</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">MultipartAutoConfiguration</span> {</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> MultipartProperties multipartProperties;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">MultipartAutoConfiguration</span><span class="params">(MultipartProperties multipartProperties)</span> {</span><br><span class="line"> <span class="built_in">this</span>.multipartProperties = multipartProperties;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Bean</span></span><br><span class="line"> <span class="meta">@ConditionalOnMissingBean({MultipartConfigElement.class, CommonsMultipartResolver.class})</span></span><br><span class="line"> <span class="keyword">public</span> MultipartConfigElement <span class="title function_">multipartConfigElement</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.multipartProperties.createMultipartConfig();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>而这个 MultipartProperties 类中</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@ConfigurationProperties(</span></span><br><span class="line"><span class="meta"> prefix = "spring.servlet.multipart",</span></span><br><span class="line"><span class="meta"> ignoreUnknownFields = false</span></span><br><span class="line"><span class="meta">)</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">MultipartProperties</span> {</span><br><span class="line"> <span class="keyword">private</span> <span class="type">boolean</span> <span class="variable">enabled</span> <span class="operator">=</span> <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">private</span> String location;</span><br><span class="line"> <span class="keyword">private</span> <span class="type">DataSize</span> <span class="variable">maxFileSize</span> <span class="operator">=</span> DataSize.ofMegabytes(<span class="number">1L</span>);</span><br><span class="line"> <span class="keyword">private</span> <span class="type">DataSize</span> <span class="variable">maxRequestSize</span> <span class="operator">=</span> DataSize.ofMegabytes(<span class="number">10L</span>);</span><br><span class="line"> <span class="keyword">private</span> <span class="type">DataSize</span> <span class="variable">fileSizeThreshold</span> <span class="operator">=</span> DataSize.ofBytes(<span class="number">0L</span>);</span><br><span class="line"> <span class="keyword">private</span> <span class="type">boolean</span> <span class="variable">resolveLazily</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br></pre></td></tr></table></figure>
|
|
<p>并且在前面 createMultipartConfig 中就使用了这个maxFileSize 的默认值</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> MultipartConfigElement <span class="title function_">createMultipartConfig</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">MultipartConfigFactory</span> <span class="variable">factory</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">MultipartConfigFactory</span>();</span><br><span class="line"> <span class="type">PropertyMapper</span> <span class="variable">map</span> <span class="operator">=</span> PropertyMapper.get().alwaysApplyingWhenNonNull();</span><br><span class="line"> map.from(<span class="built_in">this</span>.fileSizeThreshold).to(factory::setFileSizeThreshold);</span><br><span class="line"> map.from(<span class="built_in">this</span>.location).whenHasText().to(factory::setLocation);</span><br><span class="line"> map.from(<span class="built_in">this</span>.maxRequestSize).to(factory::setMaxRequestSize);</span><br><span class="line"> map.from(<span class="built_in">this</span>.maxFileSize).to(factory::setMaxFileSize);</span><br><span class="line"> <span class="keyword">return</span> factory.createMultipartConfig();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>而在 <code>org.apache.catalina.connector.Request#parseParts</code> 中，会判断 mce 的配置</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">parseParts</span><span class="params">(<span class="type">boolean</span> explicit)</span> {</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 省略一部分代码</span></span><br><span class="line"></span><br><span class="line"> <span class="type">ServletFileUpload</span> <span class="variable">upload</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ServletFileUpload</span>();</span><br><span class="line"> upload.setFileItemFactory(factory);</span><br><span class="line"> upload.setFileSizeMax(mce.getMaxFileSize());</span><br><span class="line"> upload.setSizeMax(mce.getMaxRequestSize());</span><br><span class="line"></span><br><span class="line"> parts = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> List<FileItem> items =</span><br><span class="line"> upload.parseRequest(<span class="keyword">new</span> <span class="title class_">ServletRequestContext</span>(<span class="built_in">this</span>));</span><br><span class="line"> <span class="type">int</span> <span class="variable">maxPostSize</span> <span class="operator">=</span> getConnector().getMaxPostSize();</span><br><span class="line"> <span class="type">int</span> <span class="variable">postSize</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="type">Charset</span> <span class="variable">charset</span> <span class="operator">=</span> getCharset();</span><br></pre></td></tr></table></figure>
|
|
<p>主要 org.apache.tomcat.util.http.fileupload.FileUploadBase#parseRequest</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> List<FileItem> <span class="title function_">parseRequest</span><span class="params">(<span class="keyword">final</span> RequestContext ctx)</span></span><br><span class="line"> <span class="keyword">throws</span> FileUploadException {</span><br><span class="line"> <span class="keyword">final</span> List<FileItem> items = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">successful</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">FileItemIterator</span> <span class="variable">iter</span> <span class="operator">=</span> getItemIterator(ctx);</span><br><span class="line"> <span class="keyword">final</span> <span class="type">FileItemFactory</span> <span class="variable">fileItemFactory</span> <span class="operator">=</span> Objects.requireNonNull(getFileItemFactory(),</span><br><span class="line"> <span class="string">"No FileItemFactory has been set."</span>);</span><br><span class="line"> <span class="keyword">final</span> <span class="type">byte</span>[] buffer = <span class="keyword">new</span> <span class="title class_">byte</span>[Streams.DEFAULT_BUFFER_SIZE];</span><br><span class="line"> <span class="keyword">while</span> (iter.hasNext()) {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">FileItemStream</span> <span class="variable">item</span> <span class="operator">=</span> iter.next();</span><br><span class="line"> <span class="comment">// Don't use getName() here to prevent an InvalidFileNameException.</span></span><br><span class="line"> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">fileName</span> <span class="operator">=</span> item.getName();</span><br><span class="line"> <span class="keyword">final</span> <span class="type">FileItem</span> <span class="variable">fileItem</span> <span class="operator">=</span> fileItemFactory.createItem(item.getFieldName(), item.getContentType(),</span><br><span class="line"> item.isFormField(), fileName);</span><br><span class="line"> items.add(fileItem);</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> Streams.copy(item.openStream(), fileItem.getOutputStream(), <span class="literal">true</span>, buffer);</span><br><span class="line"> } <span class="keyword">catch</span> (<span class="keyword">final</span> FileUploadIOException e) {</span><br></pre></td></tr></table></figure>
|
|
<p>其中 org.apache.tomcat.util.http.fileupload.FileUploadBase#getItemIterator</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> FileItemIterator <span class="title function_">getItemIterator</span><span class="params">(<span class="keyword">final</span> RequestContext ctx)</span></span><br><span class="line"><span class="keyword">throws</span> FileUploadException, IOException {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">FileItemIteratorImpl</span>(<span class="built_in">this</span>, ctx);</span><br><span class="line"> } <span class="keyword">catch</span> (<span class="keyword">final</span> FileUploadIOException e) {</span><br><span class="line"> <span class="comment">// unwrap encapsulated SizeException</span></span><br><span class="line"> <span class="keyword">throw</span> (FileUploadException) e.getCause();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里就创建了 org.apache.tomcat.util.http.fileupload.impl.FileItemIteratorImpl</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">FileItemIteratorImpl</span><span class="params">(<span class="keyword">final</span> FileUploadBase fileUploadBase, <span class="keyword">final</span> RequestContext requestContext)</span></span><br><span class="line"> <span class="keyword">throws</span> FileUploadException, IOException {</span><br><span class="line"> <span class="built_in">this</span>.fileUploadBase = fileUploadBase;</span><br><span class="line"> sizeMax = fileUploadBase.getSizeMax();</span><br><span class="line"> fileSizeMax = fileUploadBase.getFileSizeMax();</span><br><span class="line"> ctx = Objects.requireNonNull(requestContext, <span class="string">"requestContext"</span>);</span><br><span class="line"> skipPreamble = <span class="literal">true</span>;</span><br><span class="line"> findNextItem();</span><br><span class="line">}</span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
<p>内部使用了前面给 upload 设置的文件大小上限 upload.setFileSizeMax(mce.getMaxFileSize());</p>
|
|
<p>然后在 findNextItem 里执行了初始化</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="type">boolean</span> <span class="title function_">findNextItem</span><span class="params">()</span> <span class="keyword">throws</span> FileUploadException, IOException {</span><br><span class="line"> <span class="keyword">if</span> (eof) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (currentItem != <span class="literal">null</span>) {</span><br><span class="line"> currentItem.close();</span><br><span class="line"> currentItem = <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">final</span> <span class="type">MultipartStream</span> <span class="variable">multi</span> <span class="operator">=</span> getMultiPartStream();</span><br><span class="line"> <span class="keyword">for</span> (;;) {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">boolean</span> nextPart;</span><br><span class="line"> <span class="keyword">if</span> (skipPreamble) {</span><br><span class="line"> nextPart = multi.skipPreamble();</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> nextPart = multi.readBoundary();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (!nextPart) {</span><br><span class="line"> <span class="keyword">if</span> (currentFieldName == <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// Outer multipart terminated -> No more data</span></span><br><span class="line"> eof = <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// Inner multipart terminated -> Return to parsing the outer</span></span><br><span class="line"> multi.setBoundary(multiPartBoundary);</span><br><span class="line"> currentFieldName = <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">final</span> <span class="type">FileItemHeaders</span> <span class="variable">headers</span> <span class="operator">=</span> fileUploadBase.getParsedHeaders(multi.readHeaders());</span><br><span class="line"> <span class="keyword">if</span> (currentFieldName == <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// We're parsing the outer multipart</span></span><br><span class="line"> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">fieldName</span> <span class="operator">=</span> fileUploadBase.getFieldName(headers);</span><br><span class="line"> <span class="keyword">if</span> (fieldName != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">subContentType</span> <span class="operator">=</span> headers.getHeader(FileUploadBase.CONTENT_TYPE);</span><br><span class="line"> <span class="keyword">if</span> (subContentType != <span class="literal">null</span></span><br><span class="line"> && subContentType.toLowerCase(Locale.ENGLISH)</span><br><span class="line"> .startsWith(FileUploadBase.MULTIPART_MIXED)) {</span><br><span class="line"> currentFieldName = fieldName;</span><br><span class="line"> <span class="comment">// Multiple files associated with this field name</span></span><br><span class="line"> <span class="keyword">final</span> <span class="type">byte</span>[] subBoundary = fileUploadBase.getBoundary(subContentType);</span><br><span class="line"> multi.setBoundary(subBoundary);</span><br><span class="line"> skipPreamble = <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">fileName</span> <span class="operator">=</span> fileUploadBase.getFileName(headers);</span><br><span class="line"> currentItem = <span class="keyword">new</span> <span class="title class_">FileItemStreamImpl</span>(<span class="built_in">this</span>, fileName,</span><br><span class="line"> fieldName, headers.getHeader(FileUploadBase.CONTENT_TYPE),</span><br><span class="line"> fileName == <span class="literal">null</span>, getContentLength(headers));</span><br><span class="line"> currentItem.setHeaders(headers);</span><br><span class="line"> progressNotifier.noteItem();</span><br><span class="line"> itemValid = <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">fileName</span> <span class="operator">=</span> fileUploadBase.getFileName(headers);</span><br><span class="line"> <span class="keyword">if</span> (fileName != <span class="literal">null</span>) {</span><br><span class="line"> currentItem = <span class="keyword">new</span> <span class="title class_">FileItemStreamImpl</span>(<span class="built_in">this</span>, fileName,</span><br><span class="line"> currentFieldName,</span><br><span class="line"> headers.getHeader(FileUploadBase.CONTENT_TYPE),</span><br><span class="line"> <span class="literal">false</span>, getContentLength(headers));</span><br><span class="line"> currentItem.setHeaders(headers);</span><br><span class="line"> progressNotifier.noteItem();</span><br><span class="line"> itemValid = <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> multi.discardBodyData();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
<p>这里面就会 new 这个 FileItemStreamImpl</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">currentItem = <span class="keyword">new</span> <span class="title class_">FileItemStreamImpl</span>(<span class="built_in">this</span>, fileName,</span><br><span class="line"> fieldName, headers.getHeader(FileUploadBase.CONTENT_TYPE),</span><br><span class="line"> fileName == <span class="literal">null</span>, getContentLength(headers));</span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
<p>构造方法比较长</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">FileItemStreamImpl</span><span class="params">(<span class="keyword">final</span> FileItemIteratorImpl pFileItemIterator, <span class="keyword">final</span> String pName, <span class="keyword">final</span> String pFieldName,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> String pContentType, <span class="keyword">final</span> <span class="type">boolean</span> pFormField,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> <span class="type">long</span> pContentLength)</span> <span class="keyword">throws</span> FileUploadException, IOException {</span><br><span class="line"> fileItemIteratorImpl = pFileItemIterator;</span><br><span class="line"> name = pName;</span><br><span class="line"> fieldName = pFieldName;</span><br><span class="line"> contentType = pContentType;</span><br><span class="line"> formField = pFormField;</span><br><span class="line"> <span class="keyword">final</span> <span class="type">long</span> <span class="variable">fileSizeMax</span> <span class="operator">=</span> fileItemIteratorImpl.getFileSizeMax();</span><br><span class="line"> <span class="keyword">if</span> (fileSizeMax != -<span class="number">1</span> && pContentLength != -<span class="number">1</span></span><br><span class="line"> && pContentLength > fileSizeMax) {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">FileSizeLimitExceededException</span> <span class="variable">e</span> <span class="operator">=</span></span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">FileSizeLimitExceededException</span>(</span><br><span class="line"> String.format(<span class="string">"The field %s exceeds its maximum permitted size of %s bytes."</span>,</span><br><span class="line"> fieldName, Long.valueOf(fileSizeMax)),</span><br><span class="line"> pContentLength, fileSizeMax);</span><br><span class="line"> e.setFileName(pName);</span><br><span class="line"> e.setFieldName(pFieldName);</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">FileUploadIOException</span>(e);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// OK to construct stream now</span></span><br><span class="line"> <span class="keyword">final</span> <span class="type">ItemInputStream</span> <span class="variable">itemStream</span> <span class="operator">=</span> fileItemIteratorImpl.getMultiPartStream().newInputStream();</span><br><span class="line"> <span class="type">InputStream</span> <span class="variable">istream</span> <span class="operator">=</span> itemStream;</span><br><span class="line"> <span class="keyword">if</span> (fileSizeMax != -<span class="number">1</span>) {</span><br><span class="line"> istream = <span class="keyword">new</span> <span class="title class_">LimitedInputStream</span>(istream, fileSizeMax) {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">raiseError</span><span class="params">(<span class="keyword">final</span> <span class="type">long</span> pSizeMax, <span class="keyword">final</span> <span class="type">long</span> pCount)</span></span><br><span class="line"> <span class="keyword">throws</span> IOException {</span><br><span class="line"> itemStream.close(<span class="literal">true</span>);</span><br><span class="line"> <span class="keyword">final</span> <span class="type">FileSizeLimitExceededException</span> <span class="variable">e</span> <span class="operator">=</span></span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">FileSizeLimitExceededException</span>(</span><br><span class="line"> String.format(<span class="string">"The field %s exceeds its maximum permitted size of %s bytes."</span>,</span><br><span class="line"> fieldName, Long.valueOf(pSizeMax)),</span><br><span class="line"> pCount, pSizeMax);</span><br><span class="line"> e.setFieldName(fieldName);</span><br><span class="line"> e.setFileName(name);</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">FileUploadIOException</span>(e);</span><br><span class="line"> }</span><br><span class="line"> };</span><br><span class="line"> }</span><br><span class="line"> stream = istream;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>当 <code>fileSizeMax != 0</code> 的时候就会初始化 LimitedInputStream，这就就是会在前面的 </p>
|
|
<p><code>org.apache.tomcat.util.http.fileupload.FileUploadBase#parseRequest</code> 中</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">Streams.copy(item.openStream(), fileItem.getOutputStream(), <span class="literal">true</span>, buffer);</span><br></pre></td></tr></table></figure>
|
|
<p>这里的 item </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">final</span> <span class="type">FileItemIterator</span> <span class="variable">iter</span> <span class="operator">=</span> getItemIterator(ctx);</span><br><span class="line"> <span class="keyword">final</span> <span class="type">FileItemFactory</span> <span class="variable">fileItemFactory</span> <span class="operator">=</span> Objects.requireNonNull(getFileItemFactory(),</span><br><span class="line"> <span class="string">"No FileItemFactory has been set."</span>);</span><br><span class="line"> <span class="keyword">final</span> <span class="type">byte</span>[] buffer = <span class="keyword">new</span> <span class="title class_">byte</span>[Streams.DEFAULT_BUFFER_SIZE];</span><br><span class="line"> <span class="keyword">while</span> (iter.hasNext()) {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">FileItemStream</span> <span class="variable">item</span> <span class="operator">=</span> iter.next();</span><br></pre></td></tr></table></figure>
|
|
<p>调用了 FileItemIterator 迭代器的 next</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> FileItemStream <span class="title function_">next</span><span class="params">()</span> <span class="keyword">throws</span> FileUploadException, IOException {</span><br><span class="line"> <span class="keyword">if</span> (eof || (!itemValid && !hasNext())) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">NoSuchElementException</span>();</span><br><span class="line"> }</span><br><span class="line"> itemValid = <span class="literal">false</span>;</span><br><span class="line"> <span class="keyword">return</span> currentItem;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这个 <code>currentItem</code> 就是前面 new 的 FileItemStreamImpl</p>
|
|
<p>然后在 Streams.copy 的时候调用 openStream 也就是 <code>org.apache.tomcat.util.http.fileupload.impl.FileItemStreamImpl#openStream</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> InputStream <span class="title function_">openStream</span><span class="params">()</span> <span class="keyword">throws</span> IOException {</span><br><span class="line"> <span class="keyword">if</span> (((Closeable) stream).isClosed()) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">FileItemStream</span>.ItemSkippedException();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> stream;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里的 stream 就是 FileItemStreamImpl 构造方法最后赋值的 stream，会在大小超过限制时抛出错误</p>
|
|
<p>而这个可以通过设置 properties 来修改，spring.servlet.multipart.max-file-size 和 spring.servlet.multipart.max-request-size</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">spring.servlet.multipart.max-file-size=100MB</span><br><span class="line">spring.servlet.multipart.max-request-size=100MB</span><br></pre></td></tr></table></figure>
|
|
<p>而老版本的 <code>spring.http.multipart.maxFileSize</code><br>其实就是配置名称改了下，但是能看一下代码也是有点收获的。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
<category>Tomcat</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Tomcat</tag>
|
|
<tag>SpringBoot</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Tomcat 系列篇五-介绍下 Service 启动过程</title>
|
|
<url>/2023/10/07/Tomcat-%E7%B3%BB%E5%88%97%E7%AF%87%E5%9B%9B-%E4%BB%8B%E7%BB%8D%E4%B8%8B-Service-%E5%90%AF%E5%8A%A8%E8%BF%87%E7%A8%8B/</url>
|
|
<content><![CDATA[<p>这里开始介绍下 Service 的启动过程，Tomcat 的启动过程中</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">start</span><span class="params">()</span> <span class="keyword">throws</span> LifecycleException {</span><br><span class="line"> getServer();</span><br><span class="line"> server.start();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>getServer之前讲到过</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Server <span class="title function_">getServer</span><span class="params">()</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (server != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> server;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> System.setProperty(<span class="string">"catalina.useNaming"</span>, <span class="string">"false"</span>);</span><br><span class="line"></span><br><span class="line"> server = <span class="keyword">new</span> <span class="title class_">StandardServer</span>();</span><br><span class="line"></span><br><span class="line"> initBaseDir();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Set configuration source</span></span><br><span class="line"> ConfigFileLoader.setSource(<span class="keyword">new</span> <span class="title class_">CatalinaBaseConfigurationSource</span>(<span class="keyword">new</span> <span class="title class_">File</span>(basedir), <span class="literal">null</span>));</span><br><span class="line"></span><br><span class="line"> server.setPort( -<span class="number">1</span> );</span><br><span class="line"></span><br><span class="line"> <span class="type">Service</span> <span class="variable">service</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">StandardService</span>();</span><br><span class="line"> service.setName(<span class="string">"Tomcat"</span>);</span><br><span class="line"> server.addService(service);</span><br><span class="line"> <span class="keyword">return</span> server;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>会 new 一个StandardService，添加到 server 里，然后进行启动<br>而这个外面的 server.start 其实调用的是 <code>org.apache.catalina.Lifecycle#start</code>,<br>里面是一个 Lifecycle 的接口，这个接口被很多 Tomcat 的组件实现，其实都共用了 Lifecycle 的机制<br>然后 Lifecycle 里面会根据状态，调用实际的实现层的 startInternal 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">startInternal</span><span class="params">()</span> <span class="keyword">throws</span> LifecycleException {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span>(log.isInfoEnabled()) {</span><br><span class="line"> log.info(sm.getString(<span class="string">"standardService.start.name"</span>, <span class="built_in">this</span>.name));</span><br><span class="line"> }</span><br><span class="line"> setState(LifecycleState.STARTING);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Start our defined Container first</span></span><br><span class="line"> <span class="keyword">if</span> (engine != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">synchronized</span> (engine) {</span><br><span class="line"> engine.start();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">synchronized</span> (executors) {</span><br><span class="line"> <span class="keyword">for</span> (Executor executor: executors) {</span><br><span class="line"> executor.start();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> mapperListener.start();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Start our defined Connectors second</span></span><br><span class="line"> <span class="keyword">synchronized</span> (connectorsLock) {</span><br><span class="line"> <span class="keyword">for</span> (Connector connector: connectors) {</span><br><span class="line"> <span class="comment">// If it has already failed, don't try and start it</span></span><br><span class="line"> <span class="keyword">if</span> (connector.getState() != LifecycleState.FAILED) {</span><br><span class="line"> connector.start();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里会先设置状态，这个状态也有点东西，可以后面再讲，因为里面有个状态事件的触发<br>然后是启动 Engine，也就是 container，在这里就是 StandardEngine，先不深入去讲里面做了啥，后面讲 StandardEngine 的启动过程会讲，然后是启动线程池了，对于 StandardService 是没有线程池要启动的，或者是 springboot 集成的这个 Tomcat 中不需要，接着就是 mapperListener 的启动，这里其实是给 container 这种添加 listener，用来监听事件，做处理<br>然后就是启动 connector，connector 的启动就是之前说的，里面会启动 protocolHandler，这里还是一样的通过 Lifecycle 的接口，再通过 Lifecycle 的模板方法调用实际的 connector 实现 startInternal 方法，这也是 Tomcat 的一大特点，关于 Tomcat 也是个很大的课题，后面可能还会调整下组织结构，对新同学更友好一点。<br>值得注意的还有两个<br>第一个是添加 connector，先是锁一下，设置 connector 的 Service，然后 connectors 是个数组，这里进行了重新申请一个数组，然后进行拷贝，再把新添加的放到数组最后，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">addConnector</span><span class="params">(Connector connector)</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">synchronized</span> (connectorsLock) {</span><br><span class="line"> connector.setService(<span class="built_in">this</span>);</span><br><span class="line"> Connector results[] = <span class="keyword">new</span> <span class="title class_">Connector</span>[connectors.length + <span class="number">1</span>];</span><br><span class="line"> System.arraycopy(connectors, <span class="number">0</span>, results, <span class="number">0</span>, connectors.length);</span><br><span class="line"> results[connectors.length] = connector;</span><br><span class="line"> connectors = results;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (getState().isAvailable()) {</span><br><span class="line"> connector.start();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (LifecycleException e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalArgumentException</span>(</span><br><span class="line"> sm.getString(<span class="string">"standardService.connector.startFailed"</span>, connector), e);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Report this property change to interested listeners</span></span><br><span class="line"> support.firePropertyChange(<span class="string">"connector"</span>, <span class="literal">null</span>, connector);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后是判断当前的状态是否可用，如果可用就启动 connector, 最后触发下 connector 的变更事件<br>还有一个是设置Engine，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"></span><br><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">setContainer</span><span class="params">(Engine engine)</span> {</span><br><span class="line"> <span class="type">Engine</span> <span class="variable">oldEngine</span> <span class="operator">=</span> <span class="built_in">this</span>.engine;</span><br><span class="line"> <span class="keyword">if</span> (oldEngine != <span class="literal">null</span>) {</span><br><span class="line"> oldEngine.setService(<span class="literal">null</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">this</span>.engine = engine;</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.engine != <span class="literal">null</span>) {</span><br><span class="line"> <span class="built_in">this</span>.engine.setService(<span class="built_in">this</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (getState().isAvailable()) {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.engine != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="built_in">this</span>.engine.start();</span><br><span class="line"> } <span class="keyword">catch</span> (LifecycleException e) {</span><br><span class="line"> log.error(sm.getString(<span class="string">"standardService.engine.startFailed"</span>), e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// Restart MapperListener to pick up new engine.</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> mapperListener.stop();</span><br><span class="line"> } <span class="keyword">catch</span> (LifecycleException e) {</span><br><span class="line"> log.error(sm.getString(<span class="string">"standardService.mapperListener.stopFailed"</span>), e);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> mapperListener.start();</span><br><span class="line"> } <span class="keyword">catch</span> (LifecycleException e) {</span><br><span class="line"> log.error(sm.getString(<span class="string">"standardService.mapperListener.startFailed"</span>), e);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (oldEngine != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> oldEngine.stop();</span><br><span class="line"> } <span class="keyword">catch</span> (LifecycleException e) {</span><br><span class="line"> log.error(sm.getString(<span class="string">"standardService.engine.stopFailed"</span>), e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Report this property change to interested listeners</span></span><br><span class="line"> support.firePropertyChange(<span class="string">"container"</span>, oldEngine, <span class="built_in">this</span>.engine);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>前面两步是把第一个的 engine 的 service 设置为 null，然后第二个 engine 也就是新的这个的 service 设置成当前的 service，然后也是判断状态，启动 engine，接着是重启 mapperListener，先关闭再启动，最后是关闭老的 engine，最后的最后就是触发 container 变更事件。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
<category>Tomcat</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Tomcat</tag>
|
|
<tag>SpringBoot</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Tomcat 系列篇四-介绍下 Valve 架构</title>
|
|
<url>/2023/10/01/Tomcat-%E7%B3%BB%E5%88%97%E7%AF%87%E5%9B%9B-%E4%BB%8B%E7%BB%8D%E4%B8%8B-Valve-%E6%9E%B6%E6%9E%84/</url>
|
|
<content><![CDATA[<p>valve 是 Tomcat 架构中比较重要的一个组成部分，<br>之前说到 </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">connector.getService().getContainer().getPipeline().getFirst().invoke(</span><br><span class="line"> request, response);</span><br></pre></td></tr></table></figure>
|
|
<p>这段代码是通过 CoyoteAdapter 将请求处理往 container 传，这里就有个 pipeline 机制，这个 pipeline 可以看一下接口</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">interface</span> <span class="title class_">Pipeline</span> <span class="keyword">extends</span> <span class="title class_">Contained</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> Valve <span class="title function_">getBasic</span><span class="params">()</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">setBasic</span><span class="params">(Valve valve)</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">addValve</span><span class="params">(Valve valve)</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> Valve[] getValves();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">removeValve</span><span class="params">(Valve valve)</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> Valve <span class="title function_">getFirst</span><span class="params">()</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">isAsyncSupported</span><span class="params">()</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">findNonAsyncValves</span><span class="params">(Set<String> result)</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里可以往 pipeline 里添加 valve，然后看下 valve 的接口</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">interface</span> <span class="title class_">Valve</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> Valve <span class="title function_">getNext</span><span class="params">()</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">setNext</span><span class="params">(Valve valve)</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">backgroundProcess</span><span class="params">()</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">invoke</span><span class="params">(Request request, Response response)</span></span><br><span class="line"> <span class="keyword">throws</span> IOException, ServletException;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">isAsyncSupported</span><span class="params">()</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里主要看的就是 getNext 跟 setNext，就变成了一个有序的 pipeline，然后就是 invoke 方法，其实 pipeline 是由两部分组成，valve 在其中起到了前后衔接的重要作用，而且可以再 invoke 中进一步串联调用<br><img data-src="https://img.nicksxs.me/blog/Ttnmpu.png"><br>图中我们可以看到，对于 container 这个 pipeline，是没设置 first 的，只有 basic，basic 就是个兜底的 valve，在 StandardPipeline 中的 getFirst 实现</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> Valve <span class="title function_">getFirst</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (first != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> first;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> basic;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>取不到 first 就降级到 basic，也就是这里的 <code>StandardEngineValve</code>，<br>它的 invoke 我们来看下</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">final</span> <span class="keyword">void</span> <span class="title function_">invoke</span><span class="params">(Request request, Response response)</span></span><br><span class="line"> <span class="keyword">throws</span> IOException, ServletException {</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Select the Host to be used for this Request</span></span><br><span class="line"> <span class="type">Host</span> <span class="variable">host</span> <span class="operator">=</span> request.getHost();</span><br><span class="line"> <span class="keyword">if</span> (host == <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// HTTP 0.9 or HTTP 1.0 request without a host when no default host</span></span><br><span class="line"> <span class="comment">// is defined.</span></span><br><span class="line"> <span class="comment">// Don't overwrite an existing error</span></span><br><span class="line"> <span class="keyword">if</span> (!response.isError()) {</span><br><span class="line"> response.sendError(<span class="number">404</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (request.isAsyncSupported()) {</span><br><span class="line"> request.setAsyncSupported(host.getPipeline().isAsyncSupported());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Ask this Host to process this request</span></span><br><span class="line"> host.getPipeline().getFirst().invoke(request, response);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>比较简单，就是调用 host 中的 pipeline 里的第一个 valve 来处理<br><img data-src="https://img.nicksxs.me/blog/TOBldQ.png"><br>第一个是 org.apache.catalina.valves.ErrorReportValve，<br>这里处理的其实是先调用了 next</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">invoke</span><span class="params">(Request request, Response response)</span> <span class="keyword">throws</span> IOException, ServletException {</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Perform the request</span></span><br><span class="line"> getNext().invoke(request, response);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (response.isCommitted()) {</span><br><span class="line"> <span class="keyword">if</span> (response.setErrorReported()) {</span><br><span class="line"> <span class="comment">// Error wasn't previously reported but we can't write an error</span></span><br><span class="line"> <span class="comment">// page because the response has already been committed.</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">// See if IO is allowed</span></span><br><span class="line"> <span class="type">AtomicBoolean</span> <span class="variable">ioAllowed</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">AtomicBoolean</span>(<span class="literal">true</span>);</span><br><span class="line"> response.getCoyoteResponse().action(ActionCode.IS_IO_ALLOWED, ioAllowed);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (ioAllowed.get()) {</span><br><span class="line"> <span class="comment">// I/O is currently still allowed. Flush any data that is</span></span><br><span class="line"> <span class="comment">// still to be written to the client.</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> response.flushBuffer();</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable t) {</span><br><span class="line"> ExceptionUtils.handleThrowable(t);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// Now close immediately to signal to the client that</span></span><br><span class="line"> <span class="comment">// something went wrong</span></span><br><span class="line"> response.getCoyoteResponse().action(ActionCode.CLOSE_NOW,</span><br><span class="line"> request.getAttribute(RequestDispatcher.ERROR_EXCEPTION));</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">Throwable</span> <span class="variable">throwable</span> <span class="operator">=</span> (Throwable) request.getAttribute(RequestDispatcher.ERROR_EXCEPTION);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// If an async request is in progress and is not going to end once this</span></span><br><span class="line"> <span class="comment">// container thread finishes, do not process any error page here.</span></span><br><span class="line"> <span class="keyword">if</span> (request.isAsync() && !request.isAsyncCompleting()) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (throwable != <span class="literal">null</span> && !response.isError()) {</span><br><span class="line"> <span class="comment">// Make sure that the necessary methods have been called on the</span></span><br><span class="line"> <span class="comment">// response. (It is possible a component may just have set the</span></span><br><span class="line"> <span class="comment">// Throwable. Tomcat won't do that but other components might.)</span></span><br><span class="line"> <span class="comment">// These are safe to call at this point as we know that the response</span></span><br><span class="line"> <span class="comment">// has not been committed.</span></span><br><span class="line"> response.reset();</span><br><span class="line"> response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// One way or another, response.sendError() will have been called before</span></span><br><span class="line"> <span class="comment">// execution reaches this point and suspended the response. Need to</span></span><br><span class="line"> <span class="comment">// reverse that so this valve can write to the response.</span></span><br><span class="line"> response.setSuspended(<span class="literal">false</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> report(request, response, throwable);</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable tt) {</span><br><span class="line"> ExceptionUtils.handleThrowable(tt);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>虽然是放在 first，实际是先调用 next 的 invoke，也就是 <code>org.apache.catalina.core.StandardHostValve</code> 的invoke 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">final</span> <span class="keyword">void</span> <span class="title function_">invoke</span><span class="params">(Request request, Response response)</span></span><br><span class="line"> <span class="keyword">throws</span> IOException, ServletException {</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Select the Context to be used for this Request</span></span><br><span class="line"> <span class="type">Context</span> <span class="variable">context</span> <span class="operator">=</span> request.getContext();</span><br><span class="line"> <span class="keyword">if</span> (context == <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// Don't overwrite an existing error</span></span><br><span class="line"> <span class="keyword">if</span> (!response.isError()) {</span><br><span class="line"> response.sendError(<span class="number">404</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (request.isAsyncSupported()) {</span><br><span class="line"> request.setAsyncSupported(context.getPipeline().isAsyncSupported());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">boolean</span> <span class="variable">asyncAtStart</span> <span class="operator">=</span> request.isAsync();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> context.bind(Globals.IS_SECURITY_ENABLED, MY_CLASSLOADER);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!asyncAtStart && !context.fireRequestInitEvent(request.getRequest())) {</span><br><span class="line"> <span class="comment">// Don't fire listeners during async processing (the listener</span></span><br><span class="line"> <span class="comment">// fired for the request that called startAsync()).</span></span><br><span class="line"> <span class="comment">// If a request init listener throws an exception, the request</span></span><br><span class="line"> <span class="comment">// is aborted.</span></span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Ask this Context to process this request. Requests that are</span></span><br><span class="line"> <span class="comment">// already in error must have been routed here to check for</span></span><br><span class="line"> <span class="comment">// application defined error pages so DO NOT forward them to the the</span></span><br><span class="line"> <span class="comment">// application for processing.</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (!response.isErrorReportRequired()) {</span><br><span class="line"> <span class="comment">// 交给 context 去处理请求了</span></span><br><span class="line"> context.getPipeline().getFirst().invoke(request, response);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable t) {</span><br><span class="line"> ExceptionUtils.handleThrowable(t);</span><br><span class="line"> container.getLogger().error(<span class="string">"Exception Processing "</span> + request.getRequestURI(), t);</span><br><span class="line"> <span class="comment">// If a new error occurred while trying to report a previous</span></span><br><span class="line"> <span class="comment">// error allow the original error to be reported.</span></span><br><span class="line"> <span class="keyword">if</span> (!response.isErrorReportRequired()) {</span><br><span class="line"> request.setAttribute(RequestDispatcher.ERROR_EXCEPTION, t);</span><br><span class="line"> throwable(request, response, t);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Now that the request/response pair is back under container</span></span><br><span class="line"> <span class="comment">// control lift the suspension so that the error handling can</span></span><br><span class="line"> <span class="comment">// complete and/or the container can flush any remaining data</span></span><br><span class="line"> response.setSuspended(<span class="literal">false</span>);</span><br><span class="line"></span><br><span class="line"> <span class="type">Throwable</span> <span class="variable">t</span> <span class="operator">=</span> (Throwable) request.getAttribute(RequestDispatcher.ERROR_EXCEPTION);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Protect against NPEs if the context was destroyed during a</span></span><br><span class="line"> <span class="comment">// long running request.</span></span><br><span class="line"> <span class="keyword">if</span> (!context.getState().isAvailable()) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Look for (and render if found) an application level error page</span></span><br><span class="line"> <span class="keyword">if</span> (response.isErrorReportRequired()) {</span><br><span class="line"> <span class="comment">// If an error has occurred that prevents further I/O, don't waste time</span></span><br><span class="line"> <span class="comment">// producing an error report that will never be read</span></span><br><span class="line"> <span class="type">AtomicBoolean</span> <span class="variable">result</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">AtomicBoolean</span>(<span class="literal">false</span>);</span><br><span class="line"> response.getCoyoteResponse().action(ActionCode.IS_IO_ALLOWED, result);</span><br><span class="line"> <span class="keyword">if</span> (result.get()) {</span><br><span class="line"> <span class="keyword">if</span> (t != <span class="literal">null</span>) {</span><br><span class="line"> throwable(request, response, t);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> status(request, response);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!request.isAsync() && !asyncAtStart) {</span><br><span class="line"> context.fireRequestDestroyEvent(request.getRequest());</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="comment">// Access a session (if present) to update last accessed time, based</span></span><br><span class="line"> <span class="comment">// on a strict interpretation of the specification</span></span><br><span class="line"> <span class="keyword">if</span> (ACCESS_SESSION) {</span><br><span class="line"> request.getSession(<span class="literal">false</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> context.unbind(Globals.IS_SECURITY_ENABLED, MY_CLASSLOADER);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里的往下就是调用 context 的 pipeline 去处理请求了, StandardContext 的 pipeline 里的 first 是<br>org.apache.catalina.authenticator.NonLoginAuthenticator，处理认证相关的，然后 basic 就是<br>org.apache.catalina.core.StandardContextValve, 这里的来看下</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">final</span> <span class="keyword">void</span> <span class="title function_">invoke</span><span class="params">(Request request, Response response)</span></span><br><span class="line"> <span class="keyword">throws</span> IOException, ServletException {</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Disallow any direct access to resources under WEB-INF or META-INF</span></span><br><span class="line"> <span class="type">MessageBytes</span> <span class="variable">requestPathMB</span> <span class="operator">=</span> request.getRequestPathMB();</span><br><span class="line"> <span class="keyword">if</span> ((requestPathMB.startsWithIgnoreCase(<span class="string">"/META-INF/"</span>, <span class="number">0</span>))</span><br><span class="line"> || (requestPathMB.equalsIgnoreCase(<span class="string">"/META-INF"</span>))</span><br><span class="line"> || (requestPathMB.startsWithIgnoreCase(<span class="string">"/WEB-INF/"</span>, <span class="number">0</span>))</span><br><span class="line"> || (requestPathMB.equalsIgnoreCase(<span class="string">"/WEB-INF"</span>))) {</span><br><span class="line"> response.sendError(HttpServletResponse.SC_NOT_FOUND);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Select the Wrapper to be used for this Request</span></span><br><span class="line"> <span class="type">Wrapper</span> <span class="variable">wrapper</span> <span class="operator">=</span> request.getWrapper();</span><br><span class="line"> <span class="keyword">if</span> (wrapper == <span class="literal">null</span> || wrapper.isUnavailable()) {</span><br><span class="line"> response.sendError(HttpServletResponse.SC_NOT_FOUND);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Acknowledge the request</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> response.sendAcknowledgement(ContinueResponseTiming.IMMEDIATELY);</span><br><span class="line"> } <span class="keyword">catch</span> (IOException ioe) {</span><br><span class="line"> container.getLogger().error(sm.getString(</span><br><span class="line"> <span class="string">"standardContextValve.acknowledgeException"</span>), ioe);</span><br><span class="line"> request.setAttribute(RequestDispatcher.ERROR_EXCEPTION, ioe);</span><br><span class="line"> response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (request.isAsyncSupported()) {</span><br><span class="line"> request.setAsyncSupported(wrapper.getPipeline().isAsyncSupported());</span><br><span class="line"> }</span><br><span class="line"> wrapper.getPipeline().getFirst().invoke(request, response);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>会调用 wrapper 的 pipeline 去处理请求，这里也只有一个<br> <code>org.apache.catalina.core.StandardWrapperValve</code><br>这部分的逻辑比较长，因为要串联后面的 filter 流程</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">final</span> <span class="keyword">void</span> <span class="title function_">invoke</span><span class="params">(Request request, Response response)</span></span><br><span class="line"> <span class="keyword">throws</span> IOException, ServletException {</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Initialize local variables we may need</span></span><br><span class="line"> <span class="type">boolean</span> <span class="variable">unavailable</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="type">Throwable</span> <span class="variable">throwable</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="comment">// This should be a Request attribute...</span></span><br><span class="line"> <span class="type">long</span> t1=System.currentTimeMillis();</span><br><span class="line"> requestCount.incrementAndGet();</span><br><span class="line"> <span class="type">StandardWrapper</span> <span class="variable">wrapper</span> <span class="operator">=</span> (StandardWrapper) getContainer();</span><br><span class="line"> <span class="type">Servlet</span> <span class="variable">servlet</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="type">Context</span> <span class="variable">context</span> <span class="operator">=</span> (Context) wrapper.getParent();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Check for the application being marked unavailable</span></span><br><span class="line"> <span class="keyword">if</span> (!context.getState().isAvailable()) {</span><br><span class="line"> response.sendError(HttpServletResponse.SC_SERVICE_UNAVAILABLE,</span><br><span class="line"> sm.getString(<span class="string">"standardContext.isUnavailable"</span>));</span><br><span class="line"> unavailable = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Check for the servlet being marked unavailable</span></span><br><span class="line"> <span class="keyword">if</span> (!unavailable && wrapper.isUnavailable()) {</span><br><span class="line"> container.getLogger().info(sm.getString(<span class="string">"standardWrapper.isUnavailable"</span>,</span><br><span class="line"> wrapper.getName()));</span><br><span class="line"> <span class="type">long</span> <span class="variable">available</span> <span class="operator">=</span> wrapper.getAvailable();</span><br><span class="line"> <span class="keyword">if</span> ((available > <span class="number">0L</span>) && (available < Long.MAX_VALUE)) {</span><br><span class="line"> response.setDateHeader(<span class="string">"Retry-After"</span>, available);</span><br><span class="line"> response.sendError(HttpServletResponse.SC_SERVICE_UNAVAILABLE,</span><br><span class="line"> sm.getString(<span class="string">"standardWrapper.isUnavailable"</span>,</span><br><span class="line"> wrapper.getName()));</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (available == Long.MAX_VALUE) {</span><br><span class="line"> response.sendError(HttpServletResponse.SC_NOT_FOUND,</span><br><span class="line"> sm.getString(<span class="string">"standardWrapper.notFound"</span>,</span><br><span class="line"> wrapper.getName()));</span><br><span class="line"> }</span><br><span class="line"> unavailable = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Allocate a servlet instance to process this request</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (!unavailable) {</span><br><span class="line"> servlet = wrapper.allocate();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (UnavailableException e) {</span><br><span class="line"> container.getLogger().error(</span><br><span class="line"> sm.getString(<span class="string">"standardWrapper.allocateException"</span>,</span><br><span class="line"> wrapper.getName()), e);</span><br><span class="line"> <span class="type">long</span> <span class="variable">available</span> <span class="operator">=</span> wrapper.getAvailable();</span><br><span class="line"> <span class="keyword">if</span> ((available > <span class="number">0L</span>) && (available < Long.MAX_VALUE)) {</span><br><span class="line"> response.setDateHeader(<span class="string">"Retry-After"</span>, available);</span><br><span class="line"> response.sendError(HttpServletResponse.SC_SERVICE_UNAVAILABLE,</span><br><span class="line"> sm.getString(<span class="string">"standardWrapper.isUnavailable"</span>,</span><br><span class="line"> wrapper.getName()));</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (available == Long.MAX_VALUE) {</span><br><span class="line"> response.sendError(HttpServletResponse.SC_NOT_FOUND,</span><br><span class="line"> sm.getString(<span class="string">"standardWrapper.notFound"</span>,</span><br><span class="line"> wrapper.getName()));</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (ServletException e) {</span><br><span class="line"> container.getLogger().error(sm.getString(<span class="string">"standardWrapper.allocateException"</span>,</span><br><span class="line"> wrapper.getName()), StandardWrapper.getRootCause(e));</span><br><span class="line"> throwable = e;</span><br><span class="line"> exception(request, response, e);</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> ExceptionUtils.handleThrowable(e);</span><br><span class="line"> container.getLogger().error(sm.getString(<span class="string">"standardWrapper.allocateException"</span>,</span><br><span class="line"> wrapper.getName()), e);</span><br><span class="line"> throwable = e;</span><br><span class="line"> exception(request, response, e);</span><br><span class="line"> servlet = <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">MessageBytes</span> <span class="variable">requestPathMB</span> <span class="operator">=</span> request.getRequestPathMB();</span><br><span class="line"> <span class="type">DispatcherType</span> <span class="variable">dispatcherType</span> <span class="operator">=</span> DispatcherType.REQUEST;</span><br><span class="line"> <span class="keyword">if</span> (request.getDispatcherType()==DispatcherType.ASYNC) {</span><br><span class="line"> dispatcherType = DispatcherType.ASYNC;</span><br><span class="line"> }</span><br><span class="line"> request.setAttribute(Globals.DISPATCHER_TYPE_ATTR,dispatcherType);</span><br><span class="line"> request.setAttribute(Globals.DISPATCHER_REQUEST_PATH_ATTR,</span><br><span class="line"> requestPathMB);</span><br><span class="line"> <span class="comment">// Create the filter chain for this request</span></span><br><span class="line"> <span class="type">ApplicationFilterChain</span> <span class="variable">filterChain</span> <span class="operator">=</span></span><br><span class="line"> ApplicationFilterFactory.createFilterChain(request, wrapper, servlet);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Call the filter chain for this request</span></span><br><span class="line"> <span class="comment">// <span class="doctag">NOTE:</span> This also calls the servlet's service() method</span></span><br><span class="line"> <span class="type">Container</span> <span class="variable">container</span> <span class="operator">=</span> <span class="built_in">this</span>.container;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> ((servlet != <span class="literal">null</span>) && (filterChain != <span class="literal">null</span>)) {</span><br><span class="line"> <span class="comment">// Swallow output if needed</span></span><br><span class="line"> <span class="keyword">if</span> (context.getSwallowOutput()) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> SystemLogHandler.startCapture();</span><br><span class="line"> <span class="keyword">if</span> (request.isAsyncDispatching()) {</span><br><span class="line"> request.getAsyncContextInternal().doInternalDispatch();</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> filterChain.doFilter(request.getRequest(),</span><br><span class="line"> response.getResponse());</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">log</span> <span class="operator">=</span> SystemLogHandler.stopCapture();</span><br><span class="line"> <span class="keyword">if</span> (log != <span class="literal">null</span> && log.length() > <span class="number">0</span>) {</span><br><span class="line"> context.getLogger().info(log);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">if</span> (request.isAsyncDispatching()) {</span><br><span class="line"> request.getAsyncContextInternal().doInternalDispatch();</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> filterChain.doFilter</span><br><span class="line"> (request.getRequest(), response.getResponse());</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (ClientAbortException | CloseNowException e) {</span><br><span class="line"> <span class="keyword">if</span> (container.getLogger().isDebugEnabled()) {</span><br><span class="line"> container.getLogger().debug(sm.getString(</span><br><span class="line"> <span class="string">"standardWrapper.serviceException"</span>, wrapper.getName(),</span><br><span class="line"> context.getName()), e);</span><br><span class="line"> }</span><br><span class="line"> throwable = e;</span><br><span class="line"> exception(request, response, e);</span><br><span class="line"> } <span class="keyword">catch</span> (IOException e) {</span><br><span class="line"> container.getLogger().error(sm.getString(</span><br><span class="line"> <span class="string">"standardWrapper.serviceException"</span>, wrapper.getName(),</span><br><span class="line"> context.getName()), e);</span><br><span class="line"> throwable = e;</span><br><span class="line"> exception(request, response, e);</span><br><span class="line"> } <span class="keyword">catch</span> (UnavailableException e) {</span><br><span class="line"> container.getLogger().error(sm.getString(</span><br><span class="line"> <span class="string">"standardWrapper.serviceException"</span>, wrapper.getName(),</span><br><span class="line"> context.getName()), e);</span><br><span class="line"> <span class="comment">// throwable = e;</span></span><br><span class="line"> <span class="comment">// exception(request, response, e);</span></span><br><span class="line"> wrapper.unavailable(e);</span><br><span class="line"> <span class="type">long</span> <span class="variable">available</span> <span class="operator">=</span> wrapper.getAvailable();</span><br><span class="line"> <span class="keyword">if</span> ((available > <span class="number">0L</span>) && (available < Long.MAX_VALUE)) {</span><br><span class="line"> response.setDateHeader(<span class="string">"Retry-After"</span>, available);</span><br><span class="line"> response.sendError(HttpServletResponse.SC_SERVICE_UNAVAILABLE,</span><br><span class="line"> sm.getString(<span class="string">"standardWrapper.isUnavailable"</span>,</span><br><span class="line"> wrapper.getName()));</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (available == Long.MAX_VALUE) {</span><br><span class="line"> response.sendError(HttpServletResponse.SC_NOT_FOUND,</span><br><span class="line"> sm.getString(<span class="string">"standardWrapper.notFound"</span>,</span><br><span class="line"> wrapper.getName()));</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// Do not save exception in 'throwable', because we</span></span><br><span class="line"> <span class="comment">// do not want to do exception(request, response, e) processing</span></span><br><span class="line"> } <span class="keyword">catch</span> (ServletException e) {</span><br><span class="line"> <span class="type">Throwable</span> <span class="variable">rootCause</span> <span class="operator">=</span> StandardWrapper.getRootCause(e);</span><br><span class="line"> <span class="keyword">if</span> (!(rootCause <span class="keyword">instanceof</span> ClientAbortException)) {</span><br><span class="line"> container.getLogger().error(sm.getString(</span><br><span class="line"> <span class="string">"standardWrapper.serviceExceptionRoot"</span>,</span><br><span class="line"> wrapper.getName(), context.getName(), e.getMessage()),</span><br><span class="line"> rootCause);</span><br><span class="line"> }</span><br><span class="line"> throwable = e;</span><br><span class="line"> exception(request, response, e);</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> ExceptionUtils.handleThrowable(e);</span><br><span class="line"> container.getLogger().error(sm.getString(</span><br><span class="line"> <span class="string">"standardWrapper.serviceException"</span>, wrapper.getName(),</span><br><span class="line"> context.getName()), e);</span><br><span class="line"> throwable = e;</span><br><span class="line"> exception(request, response, e);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="comment">// Release the filter chain (if any) for this request</span></span><br><span class="line"> <span class="keyword">if</span> (filterChain != <span class="literal">null</span>) {</span><br><span class="line"> filterChain.release();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Deallocate the allocated servlet instance</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (servlet != <span class="literal">null</span>) {</span><br><span class="line"> wrapper.deallocate(servlet);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> ExceptionUtils.handleThrowable(e);</span><br><span class="line"> container.getLogger().error(sm.getString(<span class="string">"standardWrapper.deallocateException"</span>,</span><br><span class="line"> wrapper.getName()), e);</span><br><span class="line"> <span class="keyword">if</span> (throwable == <span class="literal">null</span>) {</span><br><span class="line"> throwable = e;</span><br><span class="line"> exception(request, response, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// If this servlet has been marked permanently unavailable,</span></span><br><span class="line"> <span class="comment">// unload it and release this instance</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> ((servlet != <span class="literal">null</span>) &&</span><br><span class="line"> (wrapper.getAvailable() == Long.MAX_VALUE)) {</span><br><span class="line"> wrapper.unload();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> ExceptionUtils.handleThrowable(e);</span><br><span class="line"> container.getLogger().error(sm.getString(<span class="string">"standardWrapper.unloadException"</span>,</span><br><span class="line"> wrapper.getName()), e);</span><br><span class="line"> <span class="keyword">if</span> (throwable == <span class="literal">null</span>) {</span><br><span class="line"> exception(request, response, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="type">long</span> t2=System.currentTimeMillis();</span><br><span class="line"></span><br><span class="line"> <span class="type">long</span> time=t2-t1;</span><br><span class="line"> processingTime += time;</span><br><span class="line"> <span class="keyword">if</span>( time > maxTime) {</span><br><span class="line"> maxTime=time;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span>( time < minTime) {</span><br><span class="line"> minTime=time;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里就会创建 ApplicationFilterChain 然后进行<br><code>filterChain.doFilter(request.getRequest(), response.getResponse());</code><br>doFilter 处理</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
<category>Tomcat</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Tomcat</tag>
|
|
<tag>SpringBoot</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>two sum</title>
|
|
<url>/2015/01/14/Two-Sum/</url>
|
|
<content><![CDATA[<h3 id="problem"><a href="#problem" class="headerlink" title="problem"></a>problem</h3><p>Given an array of integers, find two numbers such that they add up to a specific target number.</p>
|
|
<p>The function twoSum should return indices of the two numbers such that they add up to the target, where index1 must be less than index2. Please note that your returned answers (both index1 and index2) are not zero-based.</p>
|
|
<span id="more"></span>
|
|
<p>You may assume that each input would have exactly one solution.</p>
|
|
<p><strong>Input</strong>: numbers={2, 7, 11, 15}, target=9<br><strong>Output</strong>: index1=1, index2=2</p>
|
|
<h3 id="code"><a href="#code" class="headerlink" title="code"></a>code</h3><figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">struct</span> <span class="title class_">Node</span></span><br><span class="line">{</span><br><span class="line"> <span class="type">int</span> num, pos;</span><br><span class="line">};</span><br><span class="line"><span class="function"><span class="type">bool</span> <span class="title">cmp</span><span class="params">(Node a, Node b)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> <span class="keyword">return</span> a.num < b.num;</span><br><span class="line">}</span><br><span class="line"><span class="keyword">class</span> <span class="title class_">Solution</span> {</span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line"> <span class="function">vector<<span class="type">int</span>> <span class="title">twoSum</span><span class="params">(vector<<span class="type">int</span>> &numbers, <span class="type">int</span> target)</span> </span>{</span><br><span class="line"> <span class="comment">// Start typing your C/C++ solution below</span></span><br><span class="line"> <span class="comment">// DO NOT write int main() function</span></span><br><span class="line"> vector<<span class="type">int</span>> result;</span><br><span class="line"> vector<Node> array;</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> i = <span class="number">0</span>; i < numbers.<span class="built_in">size</span>(); i++)</span><br><span class="line"> {</span><br><span class="line"> Node temp;</span><br><span class="line"> temp.num = numbers[i];</span><br><span class="line"> temp.pos = i;</span><br><span class="line"> array.<span class="built_in">push_back</span>(temp);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">sort</span>(array.<span class="built_in">begin</span>(), array.<span class="built_in">end</span>(), cmp);</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> i = <span class="number">0</span>, j = array.<span class="built_in">size</span>() - <span class="number">1</span>; i != j;)</span><br><span class="line"> {</span><br><span class="line"> <span class="type">int</span> sum = array[i].num + array[j].num;</span><br><span class="line"> <span class="keyword">if</span> (sum == target)</span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">if</span> (array[i].pos < array[j].pos)</span><br><span class="line"> {</span><br><span class="line"> result.<span class="built_in">push_back</span>(array[i].pos + <span class="number">1</span>);</span><br><span class="line"> result.<span class="built_in">push_back</span>(array[j].pos + <span class="number">1</span>);</span><br><span class="line"> } <span class="keyword">else</span></span><br><span class="line"> {</span><br><span class="line"> result.<span class="built_in">push_back</span>(array[j].pos + <span class="number">1</span>);</span><br><span class="line"> result.<span class="built_in">push_back</span>(array[i].pos + <span class="number">1</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (sum < target)</span><br><span class="line"> {</span><br><span class="line"> i++;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (sum > target)</span><br><span class="line"> {</span><br><span class="line"> j--;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> result;</span><br><span class="line"> }</span><br><span class="line">};</span><br></pre></td></tr></table></figure>
|
|
|
|
<h3 id="Analysis"><a href="#Analysis" class="headerlink" title="Analysis"></a>Analysis</h3><p>sort the array, then test from head and end, until catch the right answer</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>c++</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Windows 莫名重启问题解决</title>
|
|
<url>/2023/12/03/Windows-%E8%8E%AB%E5%90%8D%E9%87%8D%E5%90%AF%E9%97%AE%E9%A2%98%E8%A7%A3%E5%86%B3/</url>
|
|
<content><![CDATA[<p>之前在 Windows 里用 vmware workstation 搭了个黑裙，然后硬盘直通，硬盘跑着倒还好，但是宿主机 Windows 隔一段时间就会重启，就去搜索了下，发现其实 Windows 里的事件查看器就有点像是 Linux 系统里的 dmesg 或者 journal 日志，然后根据重启的时间排查事件查看器里，Windows 日志 –> 系统，<br><img data-src="https://img.nicksxs.me/blog/yN6LTC.png"><br>然后可以在右侧 “筛选当前日志”里选择 eventlog 类型的，发现有这样一条事件<br>进程 C:\Windows\system32\svchost.exe (APP) 为用户 NT AUTHORITY\SYSTEM 开始计算机 APP 的 重新启动，原因如下: 操作系统: 恢复(计划内)<br>然后在网上搜索的时候发现有一些相关解答<br>可能的一种原因是系统在应用一些更新失败时，默认设置的策略是失败后重启，我们可以自己选择不重启，因为老是重启在我虚拟机里的黑裙没关机的情况下直接就断电重启了，还是可能会造成一些影响，<br>可以在资源管理器里右键计算机，选左侧的”高级系统设置” –> 然后在”高级” tab 下面有个”启动和故障恢复” –> “设置”<br>在弹出弹窗里将 “自动重新启动” 取消勾选<br><img data-src="https://img.nicksxs.me/blog/ZnhGJN.png"><br>这样到目前还没继续发生过重启</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Windows</category>
|
|
<category>小技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Windows</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>ambari-summary</title>
|
|
<url>/2017/05/09/ambari-summary/</url>
|
|
<content><![CDATA[<h2 id="初识ambari"><a href="#初识ambari" class="headerlink" title="初识ambari"></a>初识ambari</h2><p><a href="http://ambari.apache.org/">ambari</a>是一个大数据平台的管理工具，包含了<code>hadoop</code>, <code>yarn</code>, <code>hive</code>, <code>hbase</code>, <code>spark</code>等大数据的基础架构和工具，简化了数据平台的搭建，之前只是在同事搭建好平台后的一些使用，这次有机会从头开始用<code>ambari</code>来搭建一个测试的数据平台，过程中也踩到不少坑，简单记录下。</p>
|
|
<h2 id="简单过程"><a href="#简单过程" class="headerlink" title="简单过程"></a>简单过程</h2><ul>
|
|
<li>第一个坑<br>在刚开始是按照官网的指南，用maven构建，因为GFW的原因，导致反复失败等待，也就是这个<a href="https://cwiki.apache.org/confluence/display/AMBARI/Installation+Guide+for+Ambari+2.5.0">guide</a>，因为对maven不熟悉导致有些按图索骥，浪费了很多时间，之后才知道可以直接加repo用yum安装，然而用yum安装马上就出现了第二个坑。</li>
|
|
<li>第二个坑<br>因为在线的repo还是因为网络原因很慢很慢，用proxychains勉强把ambari-server本身安装好了，<a href="http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.5.0.3/ambari.repo">ambari.repo</a>将这个放进<code>/etc/yum.repos.d/</code>路径下，然后<code>yum update && yum install ambari-server</code>安装即可，如果有条件就用proxychains走下代理。</li>
|
|
<li>第三步<br>安装好ambari-server后先执行<code>ambari-server setup</code>做一些初始化设置，其中包含了JDK路径的设置，数据库设置，设置好就OK了，然后执行<code>ambari-server start</code>启动服务，这里有个小插曲，因为<code>ambari-server</code>涉及到这么多服务，所以管理控制监控之类的模块是必不可少的，这部分可以在<code>ambari-server</code>的web ui界面安装，也可以命令行提前安装，这部分被称为<code>HDF Management Pack</code>，运行<code>ambari-server install-mpack \ --mpack=http://public-repo-1.hortonworks.com/HDF/centos7/2.x/updates/2.1.4.0/tars/hdf_ambari_mp/hdf-ambari-mpack-2.1.4.0-5.tar.gz \ --purge \ --verbose</code><br>安装，当然这个压缩包可以下载之后指到本地路径安装，然后就可以重启<code>ambari-server</code></li>
|
|
</ul>
|
|
]]></content>
|
|
<categories>
|
|
<category>data analysis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>hadoop</tag>
|
|
<tag>cluster</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>binary-watch</title>
|
|
<url>/2016/09/29/binary-watch/</url>
|
|
<content><![CDATA[<h3 id="problem"><a href="#problem" class="headerlink" title="problem"></a>problem</h3><p>A binary watch has 4 LEDs on the top which represent the hours (0-11), and the 6 LEDs on the bottom represent the minutes (0-59). </p>
|
|
<p>Each LED represents a zero or one, with the least significant bit on the right. </p>
|
|
<p><img data-src="http://7sbp1g.com1.z0.glb.clouddn.com/Binary_clock_samui_moon.jpg?imageView2/2/w/620"></p>
|
|
<p>For example, the above binary watch reads “3:25”. </p>
|
|
<p>Given a non-negative integer n which represents the number of LEDs that are currently on, return all possible times the watch could represent. </p>
|
|
<h4 id="Example"><a href="#Example" class="headerlink" title="Example:"></a>Example:</h4><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Input: n = 1</span><br><span class="line">Return: ["1:00", "2:00", "4:00", "8:00", "0:01", "0:02", "0:04", "0:08", "0:16", "0:32"]</span><br></pre></td></tr></table></figure>
|
|
<h4 id="Note"><a href="#Note" class="headerlink" title="Note:"></a>Note:</h4><ul>
|
|
<li>The order of output does not matter.</li>
|
|
<li>The hour must not contain a leading zero, for example “01:00” is not valid, it should be “1:00”.</li>
|
|
<li>The minute must be consist of two digits and may contain a leading zero, for example “10:2” is not valid, it should be “10:02”.</li>
|
|
</ul>
|
|
<h3 id="题解"><a href="#题解" class="headerlink" title="题解"></a>题解</h3><p>又是参（chao）考（xi）别人的代码，嗯，就是这么不要脸，<a href="http://www.cnblogs.com/grandyang/p/5896454.html">链接</a></p>
|
|
<h4 id="Code"><a href="#Code" class="headerlink" title="Code"></a>Code</h4><figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">Solution</span> {</span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line"> <span class="function">vector<string> <span class="title">readBinaryWatch</span><span class="params">(<span class="type">int</span> num)</span> </span>{</span><br><span class="line"> vector<string> res;</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> h = <span class="number">0</span>; h < <span class="number">12</span>; ++h) {</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> m = <span class="number">0</span>; m < <span class="number">60</span>; ++m) {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">bitset</span><<span class="number">10</span>>((h << <span class="number">6</span>) + m).<span class="built_in">count</span>() == num) {</span><br><span class="line"> res.<span class="built_in">push_back</span>(<span class="built_in">to_string</span>(h) + (m < <span class="number">10</span> ? <span class="string">":0"</span> : <span class="string">":"</span>) + <span class="built_in">to_string</span>(m));</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> res;</span><br><span class="line"> }</span><br><span class="line">};</span><br></pre></td></tr></table></figure>]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>c++</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>dnsmasq的一个使用注意点</title>
|
|
<url>/2023/04/16/dnsmasq%E7%9A%84%E4%B8%80%E4%B8%AA%E4%BD%BF%E7%94%A8%E6%B3%A8%E6%84%8F%E7%82%B9/</url>
|
|
<content><![CDATA[<p>在本地使用了 valet 做 php 的开发环境，因为可以指定自定义域名和证书，碰巧最近公司的网络环境比较糟糕，就想要在自定义 dns 上下点功夫，本来我们经常要在 dns 那配置个内部的 dns 地址，就想是不是可以通过 dnsmasq 来解决，<br>却在第一步碰到个低级的问题，在 dnsmasq 的主配置文件里<br>我配置了解析文件路径配置<br>像这样</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">resolv-file=/opt/homebrew/etc/dnsmasq.d/resolv.dnsmasq.conf</span><br></pre></td></tr></table></figure>
|
|
<p>结果发现 dnsmasq 就起不来了，因为是 brew 服务的形式起来，发现日志也没有， dnsmasq 配置文件本身也没什么日志，这个是最讨厌的，网上搜了一圈也都没有， brew services 的服务如果启动状态是 error，并且服务本身没有日志的话就是一头雾水，并且对于 plist 来说，即使我手动加了标准输出和错误输出，<code>brew services restart</code> 的时候也是会被重新覆盖，<br>后来仔细看了下这个问题，发现它下面有这么一行配置</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">conf-dir=/opt/homebrew/etc/dnsmasq.d/,*.conf</span><br></pre></td></tr></table></figure>
|
|
<p>想了一下发现这个问题其实很简单，dnsmasq 应该是不支持同一配置文件加载两次，<br>我把 resolv 文件放在了同一个配置文件目录下，所以就被加载了两次，所以改掉目录就行了，但是目前看 dnsmasq 还不符合我的要求，也有可能我还没完全了解 dnsmasq 的使用方法，我想要的是比如按特定的域名后缀来配置对应的 dns 服务器，这样就不太会被影响，可以试试 AdGuard 看</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>dns</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>dnsmasq</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>docker-mysql-cluster</title>
|
|
<url>/2016/08/14/docker-mysql-cluster/</url>
|
|
<content><![CDATA[<h3 id="docker-mysql-cluster"><a href="#docker-mysql-cluster" class="headerlink" title="docker-mysql-cluster"></a>docker-mysql-cluster</h3><p>基于docker搭了个mysql集群，稍微记一下，<br>首先是新建mysql主库容<br> <span id="more"></span><br><code>docker run -d -e MYSQL_ROOT_PASSWORD=admin --name mysql-master -p 3307:3306 mysql:latest</code><br><code>-d</code>表示容器运行在后台，<code>-e</code>表示设置环境变量，即<code>MYSQL_ROOT_PASSWORD=admin</code>,设置了mysql的root密码，<br><code>--name</code>表示容器名，<code>-p</code>表示端口映射，将内部mysql:3306映射为外部的3307，最后的<code>mysql:latest</code>表示镜像名<br>此外还可以用<code>-v /local_path/my-master.cnf:/etc/mysql/my.cnf</code>来映射配置文件<br>然后同理启动从库<br><code>docker run -d -e MYSQL_ROOT_PASSWORD=admin --name mysql-slave -p 3308:3306 mysql:latest</code><br>然后进入主库改下配置文件<br><code>docker-enter mysql-master</code>如果无法进入就用<code>docker ps -a</code>看下容器是否在正常运行，如果status显示<br>未正常运行，则用<code>docker logs mysql-master</code>看下日志哪里出错了。<br>进入容器后，我这边使用的镜像的mysqld配置文件是在<code>/etc/mysql</code>下面，这个最新版本的mysql的配置文件包含<br>三部分，<code>/etc/mysql/my.cnf</code>和<code>/etc/mysql/conf.d/mysql.cnf</code>，还有<code>/etc/mysql/mysql.conf.d/mysqld.cnf</code><br>这里需要改的是最后一个，加上</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">log-bin = mysql-bin</span><br><span class="line">server_id = 1</span><br></pre></td></tr></table></figure>
|
|
<p>保存后退出容器重启主库容器，然后进入从库更改相同文件，</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">log-bin = mysql-bin</span><br><span class="line">server_id = 2</span><br></pre></td></tr></table></figure>
|
|
<h3 id="主从配置"><a href="#主从配置" class="headerlink" title="主从配置"></a>主从配置</h3><p>同样退出重启容器，然后是配置主从，首先进入主库，用命令mysql -u root -pxxxx进入mysql，然后赋予一个同步<br>权限<code>GRANT REPLICATION SLAVE ON *.* to 'backup'@'%' identified by '123456';</code>还是同样说明下，ON *.*表示了数<br>据库全部的权限，如果要指定数据库/表的话可以使用类似<code>testDb/testTable</code>，然后是<code>'backup'@'%'</code>表示给予同步<br>权限的用户名及其主机ip，%表示不限制ip，当然如果有防火墙的话还是会有限制的，最后的<code>identified by '123456'</code><br>表示同步用户的密码，然后就查看下主库的状态信息<code>show master status</code>，如下图：<br><img data-src="https://ooo.0o0.ooo/2016/08/10/57aac43029559.png" alt="9G5FE[9%@7%G(B`Q7]E)5@R.png"><br>把file跟position记下来，然后再开一个terminal，进入从库容器，登陆mysql，然后设置主库</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">change master to</span><br><span class="line">master_host='xxx.xxx.xxx.xxx', //如果主从库的容器都在同一个宿主机上，这里的ip是docker容器的ip</span><br><span class="line">master_user='backup', //就是上面的赋予权限的用户</span><br><span class="line">master_password='123456',</span><br><span class="line">master_log_file='mysql-bin.000004', //主库中查看到的file</span><br><span class="line">master_log_pos=312, //主库中查看到的position</span><br><span class="line">master_port=3306; //如果是同一宿主机，这里使用真实的3306端口，3308及主库的3307是给外部连接使用的</span><br></pre></td></tr></table></figure>
|
|
<p>通过<code>docker-ip mysql-master</code>可以查看容器的ip<br><img data-src="https://ooo.0o0.ooo/2016/08/10/57aac63fd02f1.png" alt="S(GP)P(M$N3~N1764@OW3E0.png"><br>这里有一点是要注意的，也是我踩的坑，就是如果是同一宿主机下两个mysql容器互联，我这里只能通过docker-ip和真实<br>的3306端口能够连接成功。<br>本文参考了<a href="http://blog.csdn.net/qq362228416/article/details/48569293">这位同学的文章</a></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>docker</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>docker</tag>
|
|
<tag>mysql</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>docker比一般多一点的初学者介绍</title>
|
|
<url>/2020/03/08/docker%E6%AF%94%E4%B8%80%E8%88%AC%E5%A4%9A%E4%B8%80%E7%82%B9%E7%9A%84%E5%88%9D%E5%AD%A6%E8%80%85%E4%BB%8B%E7%BB%8D/</url>
|
|
<content><![CDATA[<p>因为最近想搭一个phabricator用来做看板和任务管理，一开始了解这个是Easy大大有在微博推荐过，后来苏洋也在群里和博客里说到了，看上去还不错的样子，因为主角是docker所以就不介绍太多，后面有机会写一下。</p>
|
|
<p>docker最开始是之前在某位大佬的博客看到的，看上去有点神奇，感觉是一种轻量级的虚拟机，但是能做的事情好像差不多，那时候是在Ubuntu系统的vps里起一个Ubuntu的docker，然后在里面装个nginx，配置端口映射就可以访问了，后来也草草写过一篇使用docker搭建mysql集群，但是最近看了下好像是因为装docker的大佬做了一些别名还是什么操作，导致里面用的操作都不具有普遍性，而且主要是把搭的过程写了下，属于囫囵吞枣，没理解docker是干啥的，为啥用docker，就是操作了下，这几天借着搭phabricator的过程，把一些原来不理解，或者原来理解错误的地方重新理一下。</p>
|
|
<p>之前写的 mysql 集群，一主二备，这种架构在很多小型应用里都是这么配置的，而且一般是直接在三台 vps 里启动三个 mysql 实例，但是如果换成 docker 会有什么好处呢，其实就是方便部署，比如其中一台备库挂了，我要加一台，或者说备库的 qps 太高了，需要再加一个，如果要在 vps 上搭建的话，首先要买一台机器，等初始化，然后在上面修改源，更新，装 mysql ，然后配置主从，可能还要处理防火墙等等，如果把这些打包成一个 docker 镜像，并且放在自己的 docker registry，那就直接run 一下就可以了；还有比如在公司要给一个新同学整一套开发测试环境，以 Java 开发为例，要装 git，maven，jdk，配置 maven settings 和各种 rc，整合在一个镜像里的话，就会很方便了；再比如微服务的水平扩展。</p>
|
|
<p>但是为啥 docker 会有这种优势，听起来好像虚拟机也可以干这个事，但是虚拟机动辄上 G，而且需要 VMware，virtual box 等支持，不适合在Linux服务器环境使用，而且占用资源也会非常大。说得这么好，那么 docker 是啥呢</p>
|
|
<p>docker 主要使用 Linux 中已经存在的两种技术的一个整合升级，一个是 namespace，一个是cgroups，相比于虚拟机需要完整虚拟出一个操作系统运行基础，docker 基于宿主机内核，通过 namespace 和 cgroups 分隔进程，理念就是提供一个隔离的最小化运行依赖，这样子相对于虚拟机就有了巨大的便利性，具体的 namespace 和 cgroups 就先不展开讲，可以参考耗子叔的文章</p>
|
|
<h2 id="安装"><a href="#安装" class="headerlink" title="安装"></a>安装</h2><p>那么我们先安装下 docker，参考官方的教程，<a href="https://docs.docker.com/install/linux/docker-ce/ubuntu/">安装</a>，我的系统是 ubuntu 的，就贴了 ubuntu 的链接，用其他系统的可以找到对应的系统文档安装，安装完了的话看看 docker 的信息</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sudo docker info</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>输出以下信息<br><img data-src="https://i.loli.net/2020/03/08/IJP157hLqlgo4Ow.png"></p>
|
|
<h2 id="简单运行"><a href="#简单运行" class="headerlink" title="简单运行"></a>简单运行</h2><p>然后再来运行个 hello world 呗，</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sudo docker run hello-world</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>输出了这些<br><img data-src="https://i.loli.net/2020/03/08/7dO2JWbf3wVLPsN.png"></p>
|
|
<p>看看这个运行命令是怎么用的，一般都会看到这样子的，sudo docker run -it ubuntu bash<code>, 前面的 docker run 反正就是运行一个容器的意思，</code>-it<code>是啥呢，还有这个什么 ubuntu bash，来看看</code>docker run`的命令帮助信息</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">-i, --interactive Keep STDIN open even if not attached</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>就是要有输入，我们运行的时候能输入</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">-t, --tty Allocate a pseudo-TTY</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>要有个虚拟终端，</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]</span><br><span class="line"></span><br><span class="line">Run a command in a new container</span><br></pre></td></tr></table></figure>
|
|
|
|
<h2 id="镜像"><a href="#镜像" class="headerlink" title="镜像"></a>镜像</h2><p>上面说的-it 就是这里的 options，后面那个 ubuntu 就是 image 辣，image 是啥呢</p>
|
|
<p><strong>Docker 把应用程序及其依赖，打包在 image 文件里面</strong>，可以把它理解成为类似于虚拟机的镜像或者运行一个进程的代码，跑起来了的叫docker 容器或者进程，比如我们将要运行的<code>docker run -it ubuntu bash</code>的ubuntu 就是个 ubuntu 容器的镜像，将这个镜像运行起来后，我们可以进入容器像使用 ubuntu 一样使用它，来看下我们的镜像，使用<code>sudo docker image ls</code>就能列出我们宿主机上的 docker 镜像了</p>
|
|
<p><img data-src="https://i.loli.net/2020/03/08/G3ltzyVLqheDRgo.png"></p>
|
|
<p>一个 ubuntu 镜像才 64MB，非常小巧，然后是后面的<code>bash</code>，我通过交互式启动了一个 ubuntu 容器，然后在这个启动的容器里运行了 bash 命令，这样就可以在容器里玩一下了</p>
|
|
<h2 id="在容器里看下进程，"><a href="#在容器里看下进程，" class="headerlink" title="在容器里看下进程，"></a>在容器里看下进程，</h2><p><img data-src="https://i.loli.net/2020/03/08/2qQFPbxB9uEzcrJ.png"></p>
|
|
<p>只有刚才运行容器的 bash 进程和我刚执行的 ps，这里有个可以注意下的，bash 这个进程的 pid 是 1，其实这里就用到了 linux 中的PID Namespace，容器会隔离出一个 pid 的名字空间，这里面的进程跟外部的 pid 命名独立</p>
|
|
<h2 id="查看宿主机上的容器"><a href="#查看宿主机上的容器" class="headerlink" title="查看宿主机上的容器"></a>查看宿主机上的容器</h2><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sudo docker ps -a</span><br></pre></td></tr></table></figure>
|
|
|
|
<p><img data-src="https://i.loli.net/2020/03/08/spGtzcDXKv3ORU7.png"></p>
|
|
<h2 id="如何进入一个正在运行中的-docker-容器"><a href="#如何进入一个正在运行中的-docker-容器" class="headerlink" title="如何进入一个正在运行中的 docker 容器"></a>如何进入一个正在运行中的 docker 容器</h2><p>这个应该是比较常用的，因为比如是一个微服务容器，有时候就像看下运行状态，日志啥的</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sudo docker exec -it [containerID] bash</span><br></pre></td></tr></table></figure>
|
|
|
|
<p><img data-src="https://i.loli.net/2020/03/08/uswmh3Izp65kc9n.png"></p>
|
|
<h2 id="查看日志"><a href="#查看日志" class="headerlink" title="查看日志"></a>查看日志</h2><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sudo docker logs [containerID]</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>我在运行容器的终端里胡乱输入点啥，然后通过上面的命令就可以看到啦</p>
|
|
<p><img data-src="https://i.loli.net/2020/03/08/RJAvlhEMD7VfbXz.png"></p>
|
|
<p><img data-src="https://i.loli.net/2020/03/08/L4YRni95lBzEmFh.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Docker</category>
|
|
<category>介绍</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Docker</tag>
|
|
<tag>namespace</tag>
|
|
<tag>cgroup</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 572 Subtree of Another Tree 题解分析</title>
|
|
<url>/2024/01/28/Leetcode-572-Subtree-of-Another-Tree-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<h2 id="题目介绍"><a href="#题目介绍" class="headerlink" title="题目介绍"></a>题目介绍</h2><p>Given the roots of two binary trees root and subRoot, return true if there is a subtree of root with the same structure and node values of subRoot and false otherwise.</p>
|
|
<p>A subtree of a binary tree tree is a tree that consists of a node in tree and all of this node’s descendants. The tree tree could also be considered as a subtree of itself.</p>
|
|
<h2 id="示例-1"><a href="#示例-1" class="headerlink" title="示例 1"></a>示例 1</h2><p><img data-src="https://img.nicksxs.me/blog/3JTlNY.png"></p>
|
|
<blockquote>
|
|
<p>Input: root = [3,4,5,1,2], subRoot = [4,1,2]<br>Output: true</p>
|
|
</blockquote>
|
|
<h2 id="示例-2"><a href="#示例-2" class="headerlink" title="示例 2"></a>示例 2</h2><p><img data-src="https://img.nicksxs.me/blog/hnatjE.png"></p>
|
|
<blockquote>
|
|
<p>Input: root = [3,4,5,1,2,null,null,null,null,0], subRoot = [4,1,2]<br>Output: false</p>
|
|
</blockquote>
|
|
<h2 id="解析"><a href="#解析" class="headerlink" title="解析"></a>解析</h2><p>需要判断subRoot 为根节点的这棵树是不是 root 为根节点的这个树的子树，题应该不是特别难理解，只是需要注意，这里是子树，而不是一个子结构，也就是示例 2 的判断结果是 false，因为左边子树还有个 0，只能认为是左边的一部分，但不是子树，思路分析也不难，就是递归去判断，只不过是两种维度，第一种就是找到子树的根节点，也就是在 root 树找到 subRoot 的根节点相同的节点，这个就会用到递归，如果找到了一个，就开始递归的对比所有子节点，必须完全一样</p>
|
|
<h2 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">isSubtree</span><span class="params">(TreeNode root, TreeNode subRoot)</span> {</span><br><span class="line"> <span class="keyword">if</span> (subRoot == <span class="literal">null</span> || root == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (root.val == subRoot.val) {</span><br><span class="line"> <span class="comment">// 如果 root 节点跟 subRoot 节点相同，就开始递归对比所有的子节点</span></span><br><span class="line"> <span class="keyword">if</span> (isSameTree(root.left, subRoot.left) && isSameTree(root.right, subRoot.right)) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 如果 root 节点不同，就开始递归往下找与 subRoot 相同的起始节点，这里是或的关系，因为在左右子树任一找到一个就行</span></span><br><span class="line"> <span class="keyword">return</span> isSubtree(root.left, subRoot) || isSubtree(root.right, subRoot);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 需要完全相同，也就是说不能像示例 2 那样多一个子节点这种，以及其他不同的情况，反正就是要完全一样</span></span><br><span class="line"> <span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">isSameTree</span><span class="params">(TreeNode root, TreeNode subTree)</span> {</span><br><span class="line"> <span class="comment">// 如果递归到这个节点都为空也认为是相同的</span></span><br><span class="line"> <span class="keyword">if</span> (root == <span class="literal">null</span> && subTree == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 因为有前面的判断，这里就隐含了 subTree != null 这个条件</span></span><br><span class="line"> <span class="keyword">if</span> (root == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 也因为有前面的判断，这里就隐含了 root != null 这个条件</span></span><br><span class="line"> <span class="keyword">if</span> (subTree == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 两个都不为空，但是值不相同，也是 false</span></span><br><span class="line"> <span class="keyword">if</span> (root.val != subTree.val) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 跟节点相同，就开始递归判断左右子树，必须两者都相同</span></span><br><span class="line"> <span class="keyword">return</span> isSameTree(root.left, subTree.left) && isSameTree(root.right, subTree.right);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>docker比一般多一点的初学者介绍二</title>
|
|
<url>/2020/03/15/docker%E6%AF%94%E4%B8%80%E8%88%AC%E5%A4%9A%E4%B8%80%E7%82%B9%E7%9A%84%E5%88%9D%E5%AD%A6%E8%80%85%E4%BB%8B%E7%BB%8D%E4%BA%8C/</url>
|
|
<content><![CDATA[<h2 id="限制下-docker-的-cpu-使用率"><a href="#限制下-docker-的-cpu-使用率" class="headerlink" title="限制下 docker 的 cpu 使用率"></a>限制下 docker 的 cpu 使用率</h2><p>这里我们开始玩一点有意思的，我们在容器里装下 vim 和 gcc，然后写这样一段 c 代码</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><stdio.h></span></span></span><br><span class="line"><span class="type">int</span> <span class="title function_">main</span><span class="params">(<span class="type">void</span>)</span></span><br><span class="line">{</span><br><span class="line"> <span class="type">int</span> i = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">for</span>(;;) i++;</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>就是一个最简单的死循环，然后在容器里跑起来</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_">$ </span><span class="language-bash">gcc 1.c</span> </span><br><span class="line"><span class="meta prompt_">$ </span><span class="language-bash">./a.out</span></span><br></pre></td></tr></table></figure>
|
|
<p>然后我们来看下系统资源占用（CPU）<br><img data-src="https://i.loli.net/2020/03/09/Xs562iawhHyMxeO.png" alt="Xs562iawhHyMxeO"><br>上图是在容器里的，可以看到 cpu 已经 100%了<br>然后看看容器外面的<br><img data-src="https://i.loli.net/2020/03/09/ecqH8XJ4k7rKhzu.png" alt="ecqH8XJ4k7rKhzu"><br>可以看到一个核的 cpu 也被占满了，因为是个双核的机器，并且代码是单线程的<br>然后呢我们要做点啥<br>因为已经在这个 ubuntu 容器中装了 vim 和 gcc，考虑到国内的网络，所以我们先把这个容器 commit 一下，</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">docker commit -a "nick" -m "my ubuntu" f63c5607df06 my_ubuntu:v1</span><br></pre></td></tr></table></figure>
|
|
<p>然后再运行起来</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">docker run -it --cpus=0.1 my_ubuntu:v1 bash</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://i.loli.net/2020/03/15/x2oGCPQr9Mm8ZYj.png"><br>我们的代码跟可执行文件都还在，要的就是这个效果，然后再运行一下<br><img data-src="https://i.loli.net/2020/03/10/3EgzYxpqlwobNRj.png"><br>结果是这个样子的，有点神奇是不，关键就在于 run 的时候的<code>--cpus=0.1</code>这个参数，它其实就是基于我前一篇说的 cgroup 技术，能将进程之间的cpu，内存等资源进行隔离</p>
|
|
<h2 id="开始第一个-Dockerfile"><a href="#开始第一个-Dockerfile" class="headerlink" title="开始第一个 Dockerfile"></a>开始第一个 Dockerfile</h2><p>上一面为了复用那个我装了 vim 跟 gcc 的容器，我把它提交到了本地，使用了<code>docker commit</code>命令，有点类似于 git 的 commit，但是这个不是个很好的操作方式，需要手动介入，这里更推荐使用 Dockerfile 来构建镜像</p>
|
|
<figure class="highlight docker"><table><tr><td class="code"><pre><span class="line"><span class="keyword">From</span> ubuntu:latest</span><br><span class="line"><span class="keyword">MAINTAINER</span> Nicksxs <span class="string">"nicksxs@hotmail.com"</span></span><br><span class="line"><span class="keyword">RUN</span><span class="language-bash"> sed -i s@/archive.ubuntu.com/@/mirrors.aliyun.com/@g /etc/apt/sources.list</span></span><br><span class="line"><span class="keyword">RUN</span><span class="language-bash"> apt-get clean</span></span><br><span class="line"><span class="keyword">RUN</span><span class="language-bash"> apt-get update && apt install -y nginx</span></span><br><span class="line"><span class="keyword">RUN</span><span class="language-bash"> <span class="built_in">echo</span> <span class="string">'Hi, i am in container'</span> \</span></span><br><span class="line"><span class="language-bash"> > /usr/share/nginx/html/index.html</span></span><br><span class="line"><span class="keyword">EXPOSE</span> <span class="number">80</span></span><br></pre></td></tr></table></figure>
|
|
<p>先解释下这是在干嘛，首先是这个<code>From ubuntu:latest</code>基于的 ubuntu 的最新版本的镜像，然后第二行是维护人的信息，第三四行么作为墙内人你懂的，把 ubuntu 的源换成阿里云的，不然就有的等了，然后就是装下 nginx，往默认的 nginx 的入口 html 文件里输入一行欢迎语，然后暴露 80 端口<br>然后我们使用<code>sudo docker build -t="nicksxs/static_web" .</code>命令来基于这个 Dockerfile 构建我们自己的镜像，过程中是这样的<br><img data-src="https://i.loli.net/2020/03/15/aiIMrhy9WHetDSl.png"><br><img data-src="https://i.loli.net/2020/03/11/RC6yjImFZps4HWl.png"><br>可以看到图中，我的 Dockerfile 是 7 行，里面就执行了 7 步，并且每一步都有一个类似于容器 id 的层 id 出来，这里就是一个比较重要的东西，docker 在构建的时候其实是有这个层的概念，Dockerfile 里的每一行都会往上加一层，这里有还注意下命令后面的<code>.</code>，代表当前目录下会自行去寻找 Dockerfile 进行构建，构建完了之后我们再看下我们的本地镜像<br><img data-src="https://i.loli.net/2020/03/11/op5c7UGTbhjwPOI.png"><br>我们自己的镜像出现啦<br>然后有个问题，如果这个构建中途报了错咋办呢，来试试看，我们把 nginx 改成随便的一个错误名，nginxx(不知道会不会运气好真的有这玩意)，再来 build 一把<br><img data-src="https://i.loli.net/2020/03/14/ALWIobjchnu1Rvi.png"><br>找不到 nginxx 包，是不是这个镜像就完全不能用呢，当然也不是，因为前面说到了，docker 是基于层去构建的，可以看到前面的 4 个 step 都没报错，那我们基于最后的成功步骤创建下容器看看<br>也就是<code>sudo docker run -t -i bd26f991b6c8 /bin/bash</code><br>答案是可以的，只是没装成功 nginx<br><img data-src="https://i.loli.net/2020/03/14/grcMNVxTabDPipu.png"><br>还有一点注意到没，前面的几个 step 都有一句 <code>Using cache</code>，说明 docker 在构建镜像的时候是有缓存的，这也更能说明 docker 是基于层去构建镜像，同样的底包，同样的步骤，这些层是可以被复用的，这就是 docker 的构建缓存，当然我们也可以在 build 的时候加上<code>--no-cache</code>去把构建缓存禁用掉。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Docker</category>
|
|
<category>介绍</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Docker</tag>
|
|
<tag>namespace</tag>
|
|
<tag>cgroup</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>dubbo 客户端配置的一个重要知识点</title>
|
|
<url>/2022/06/11/dubbo-%E5%AE%A2%E6%88%B7%E7%AB%AF%E9%85%8D%E7%BD%AE%E7%9A%84%E4%B8%80%E4%B8%AA%E9%87%8D%E8%A6%81%E7%9F%A5%E8%AF%86%E7%82%B9/</url>
|
|
<content><![CDATA[<p>在配置项目中其实会留着比较多的问题，由于不同的项目没有比较统一的规划和框架模板，一般都是只有创建者会比较了解（可能也不了解），譬如前阵子在配置一个 springboot + dubbo 的项目，发现了dubbo 连接注册中间客户端的问题，这里可以结合下代码来看<br>比如有的应用是用的这个</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.apache.curator<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>curator-client<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>${curator.version}<span class="tag"></<span class="name">version</span>></span> </span><br><span class="line"><span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"><span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.apache.curator<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>curator-recipes<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>${curator.version}<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"><span class="tag"></<span class="name">dependency</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>有个别应用用的是这个</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>com.101tec<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>zkclient<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>0.11<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"><span class="tag"></<span class="name">dependency</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>还有的应用是找不到相关的依赖，并且这些的使用没有个比较好的说明，为啥用前者，为啥用后者，有啥注意点，<br>首先在使用 2.6.5 的 alibaba 的 dubbo 的时候，只使用后者是会报错的，至于为啥会报错，其实就是这篇文章想说明的点<br>报错的内容其实很简单, 就是缺少这个 <code>org.apache.curator.framework.CuratorFrameworkFactory</code> 类<br>这个类看着像是依赖上面的配置，但是应该不需要两个配置一块用的，所以还是需要去看代码<br>通过找上面类被依赖的和 dubbo 连接注册中心相关的代码，看到了这段指点迷津的代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@SPI("curator")</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">interface</span> <span class="title class_">ZookeeperTransporter</span> {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Adaptive({Constants.CLIENT_KEY, Constants.TRANSPORTER_KEY})</span></span><br><span class="line"> ZookeeperClient <span class="title function_">connect</span><span class="params">(URL url)</span>;</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>众所周知，dubbo 创造了叫<a href="https://nicksxs.me/2020/06/06/%E8%81%8A%E8%81%8A-Dubbo-%E7%9A%84-SPI-%E7%BB%AD%E4%B9%8B%E8%87%AA%E9%80%82%E5%BA%94%E6%8B%93%E5%B1%95/">自适应扩展点加载</a>的神奇技术，这里的 adaptive 注解中的<code>Constants.CLIENT_KEY</code> 和 <code>Constants.TRANSPORTER_KEY</code> 可以在配置 dubbo 的注册信息的时候进行配置，如果是通过 xml 配置的话，可以在 <code><dubbo:registry/></code> 这个 tag 中的以上两个 key 进行配置，<br>具体在 dubbo.xsd 中有描述</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">xsd:element</span> <span class="attr">name</span>=<span class="string">"registry"</span> <span class="attr">type</span>=<span class="string">"registryType"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">xsd:annotation</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">xsd:documentation</span>></span><![CDATA[ The registry config ]]><span class="tag"></<span class="name">xsd:documentation</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">xsd:annotation</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">xsd:element</span>></span></span><br></pre></td></tr></table></figure>
|
|
|
|
<p><img data-src="https://img.nicksxs.me/uPic/7UQxDT.jpg"><br>并且在 spi 的配置<code>com.alibaba.dubbo.remoting.zookeeper.ZookeeperTransporter</code> 中可以看到</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">zkclient=com.alibaba.dubbo.remoting.zookeeper.zkclient.ZkclientZookeeperTransporter</span><br><span class="line">curator=com.alibaba.dubbo.remoting.zookeeper.curator.CuratorZookeeperTransporter</span><br><span class="line"></span><br><span class="line">zkclient=com.alibaba.dubbo.remoting.zookeeper.zkclient.ZkclientZookeeperTransporter</span><br><span class="line">curator=com.alibaba.dubbo.remoting.zookeeper.curator.CuratorZookeeperTransporter</span><br><span class="line"></span><br><span class="line">zkclient=com.alibaba.dubbo.remoting.zookeeper.zkclient.ZkclientZookeeperTransporter</span><br><span class="line">curator=com.alibaba.dubbo.remoting.zookeeper.curator.CuratorZookeeperTransporter</span><br></pre></td></tr></table></figure>
|
|
<p>而在上面的代码里默认的SPI 值是 <code>curator</code>，所以如果不配置，那就会报上面找不到类的问题，所以如果需要使用 zkclient 的，就需要在<code><dubbo:registry/></code> 配置中添加 <code>client="zkclient"</code>这个配置，所以有些地方还是需要懂一些更深层次的原理，但也不至于每个东西都要抠到每一行代码原理，除非就是专门做这一块的。<br>还有一点是发现有些应用是碰运气，刚好有个三方包把这个类带进来了，但是这个应用就没有单独配置这块，如果不了解或者后续忘了再来查问题就会很奇怪</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Dubbo</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Dubbo</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>docker比一般多一点的初学者介绍三</title>
|
|
<url>/2020/03/21/docker%E6%AF%94%E4%B8%80%E8%88%AC%E5%A4%9A%E4%B8%80%E7%82%B9%E7%9A%84%E5%88%9D%E5%AD%A6%E8%80%85%E4%BB%8B%E7%BB%8D%E4%B8%89/</url>
|
|
<content><![CDATA[<h2 id="运行第一个-Dockerfile"><a href="#运行第一个-Dockerfile" class="headerlink" title="运行第一个 Dockerfile"></a>运行第一个 Dockerfile</h2><p>上一篇的 Dockerfile 我们停留在构建阶段，现在来把它跑起来</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">docker run -d -p 80 --name static_web nicksxs/static_web \</span><br><span class="line">nginx -g "daemon off;"</span><br></pre></td></tr></table></figure>
|
|
<p>这里的<code>-d</code>表示以分离模型运行docker (detached),然后-p 是表示将容器的 80 端口开放给宿主机，然后容器名就叫 static_web,使用了我们上次构建的 static_web 镜像，后面的是让 nginx 在前台运行<br><img data-src="https://i.loli.net/2020/03/21/VCEuf5BTopXNkcA.png"><br>可以看到返回了个容器 id，但是具体情况没出现，也没连上去，那我们想看看怎么访问在 Dockerfile 里写的静态页面，我们来看下docker 进程<br><img data-src="https://i.loli.net/2020/03/21/Cx1JYGBrHRqojpz.png"><br>发现为我们随机分配了一个宿主机的端口，32768，去服务器的防火墙把这个外网端口开一下，看看是不是符合我们的预期呢<br><img data-src="https://i.loli.net/2020/03/21/xhAcQlX5iP9KnYB.png"><br>好像不太对额，应该是 ubuntu 安装的 nginx 的默认工作目录不对，我们来进容器看看，再熟悉下命令<code>docker exec -it 4792455ca2ed /bin/bash</code><br>记得容器 id 换成自己的，进入容器后得找找 nginx 的配置文件，通常在<code>/etc/nginx</code>,<code>/usr/local/etc</code>等目录下，然后找到我们的目录是在这<br><img data-src="https://i.loli.net/2020/03/21/wE8TfkbC2d9pNuv.png"><br>所以把刚才的内容复制过去再试试<br><img data-src="https://i.loli.net/2020/03/21/qNkpHn3GW8aiwIr.png"><br>目标达成，give me five✌️</p>
|
|
<h2 id="第二个-Dockerfile"><a href="#第二个-Dockerfile" class="headerlink" title="第二个 Dockerfile"></a>第二个 Dockerfile</h2><p>然后就想来动态一点的，毕竟写过 PHP，就来试试 PHP<br>再建一个目录叫 dynamic_web，里面创建 src 目录，放一个 index.php<br>内容是</p>
|
|
<figure class="highlight php"><table><tr><td class="code"><pre><span class="line"><span class="meta"><?php</span></span><br><span class="line"><span class="keyword">echo</span> <span class="string">"Hello World!"</span>;</span><br></pre></td></tr></table></figure>
|
|
<p>然后在 dynamic_web 目录下创建 Dockerfile，</p>
|
|
<figure class="highlight dockerfile"><table><tr><td class="code"><pre><span class="line"><span class="keyword">FROM</span> trafex/alpine-nginx-php7:latest</span><br><span class="line"><span class="keyword">COPY</span><span class="language-bash"> src/ /var/www/html</span></span><br><span class="line"><span class="keyword">EXPOSE</span> <span class="number">80</span></span><br></pre></td></tr></table></figure>
|
|
<p>Dockerfile 虽然只有三行，不过要着重说明下，这个底包其实不是docker 官方的，有两点考虑，一点是官方的基本都是 php apache 的镜像，还有就是 alpine这个，截取一段中文介绍</p>
|
|
<blockquote>
|
|
<p>Alpine 操作系统是一个面向安全的轻型 Linux 发行版。它不同于通常 Linux 发行版，Alpine 采用了 musl libc 和 busybox 以减小系统的体积和运行时资源消耗，但功能上比 busybox 又完善的多，因此得到开源社区越来越多的青睐。在保持瘦身的同时，Alpine 还提供了自己的包管理工具 apk，可以通过 <a href="https://pkgs.alpinelinux.org/packages">https://pkgs.alpinelinux.org/packages</a> 网站上查询包信息，也可以直接通过 apk 命令直接查询和安装各种软件。<br>Alpine 由非商业组织维护的，支持广泛场景的 Linux发行版，它特别为资深/重度Linux用户而优化，关注安全，性能和资源效能。Alpine 镜像可以适用于更多常用场景，并且是一个优秀的可以适用于生产的基础系统/环境。</p>
|
|
</blockquote>
|
|
<blockquote>
|
|
<p>Alpine Docker 镜像也继承了 Alpine Linux 发行版的这些优势。相比于其他 Docker 镜像，它的容量非常小，仅仅只有 5 MB 左右（对比 Ubuntu 系列镜像接近 200 MB），且拥有非常友好的包管理机制。官方镜像来自 docker-alpine 项目。</p>
|
|
</blockquote>
|
|
<blockquote>
|
|
<p>目前 Docker 官方已开始推荐使用 Alpine 替代之前的 Ubuntu 做为基础镜像环境。这样会带来多个好处。包括镜像下载速度加快，镜像安全性提高，主机之间的切换更方便，占用更少磁盘空间等。 </p>
|
|
</blockquote>
|
|
<p>一方面在没有镜像的情况下，拉取 docker 镜像还是比较费力的，第二个就是也能节省硬盘空间，所以目前有大部分的 docker 镜像都将 alpine 作为基础镜像了<br>然后再来构建下<br><img data-src="https://i.loli.net/2020/03/21/56cuKf1ObvkBSG3.png"><br>这里还有个点，就是上面的那个镜像我们也是 EXPOSE 80端口，然后外部宿主机会随机映射一个端口，为了偷个懒，我们就直接指定外部端口了<br><code>docker run -d -p 80:80 dynamic_web</code>打开浏览器发现访问不了，咋回事呢<br>因为我们没看<code>trafex/alpine-nginx-php7:latest</code>这个镜像<a href="https://hub.docker.com/r/trafex/alpine-nginx-php7">说明</a>,它内部的服务是 8080 端口的，所以我们映射的暴露端口应该是 8080，再用<code> docker run -d -p 80:8080 dynamic_web</code>这个启动，<br><img data-src="https://i.loli.net/2020/03/21/qDd2LVhz1Po4McX.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Docker</category>
|
|
<category>介绍</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Docker</tag>
|
|
<tag>namespace</tag>
|
|
<tag>Dockerfile</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>docker比一般多一点的初学者介绍四</title>
|
|
<url>/2022/12/25/docker%E6%AF%94%E4%B8%80%E8%88%AC%E5%A4%9A%E4%B8%80%E7%82%B9%E7%9A%84%E5%88%9D%E5%AD%A6%E8%80%85%E4%BB%8B%E7%BB%8D%E5%9B%9B/</url>
|
|
<content><![CDATA[<p>这次单独介绍下docker体系里非常重要的cgroup，docker对资源的限制也是基于cgroup构建的，<br>简单尝试<br>新建一个shell脚本</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_">#</span><span class="language-bash">!/bin/bash</span></span><br><span class="line">while true;do</span><br><span class="line"> echo "1"</span><br><span class="line">done</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>直接执行的话就是单核100%的cpu<br><img data-src="https://img.nicksxs.me/blog/rFp7D1.jpg"></p>
|
|
<p>首先在cgroup下面建个目录</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">mkdir -p /sys/fs/cgroup/cpu/sxs_test/</span><br></pre></td></tr></table></figure>
|
|
<p>查看目录下的文件<br><img data-src="https://img.nicksxs.me/blog/b6UwLR.jpg"><br>其中cpuacct开头的表示cpu相关的统计信息，<br>我们要配置cpu的额度，是在cpu.cfs_quota_us中</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">echo 2000 > /sys/fs/cgroup/cpu/sxs_test/cpu.cfs_quota_us </span><br></pre></td></tr></table></figure>
|
|
<p>这样表示可以使用2%的cpu，总的配额是在cpu.cfs_period_us中<br><img data-src="https://img.nicksxs.me/blog/oikqB8.png"></p>
|
|
<p>然后将当前进程输入到cgroup.procs,</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">echo $$ > /sys/fs/cgroup/cpu/sxs_test/cgroup.procs</span><br></pre></td></tr></table></figure>
|
|
<p>这样就会自动继承当前进程产生的新进程<br>再次执行就可以看到cpu被限制了<br><img data-src="https://img.nicksxs.me/blog/W0ptVg.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Docker</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Docker</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>docker使用中发现的echo命令的一个小技巧及其他</title>
|
|
<url>/2020/03/29/echo%E5%91%BD%E4%BB%A4%E7%9A%84%E4%B8%80%E4%B8%AA%E5%B0%8F%E6%8A%80%E5%B7%A7/</url>
|
|
<content><![CDATA[<h2 id="echo-实操技巧"><a href="#echo-实操技巧" class="headerlink" title="echo 实操技巧"></a>echo 实操技巧</h2><p>最近做 docker 系列，会经常需要进到 docker 内部，如上一篇介绍的，这些镜像一般都有用 ubuntu 或者alpine 这样的 Linux 系统作为底包，如果构建镜像的时候没有替换源的话，因为特殊的网络原因，在内部想编辑下东西要安装个类似于 vim 这样的编辑器就会很慢很慢，像视频里 two thousand years later~ 而且如果在容器内部想改源配置的话也要编辑器，就陷入了一个鸡生蛋，跟蛋生鸡的死锁问题中，对于 linux 大神来说应该有一万种方法解决这个问题，对于我这个渣渣来说可能只想到了这个土方法，先 cp backup 一下 sources.list, 再 echo “xxx” > sources.list, 这里就碰到了一个问题，这个 sources.list 一般不止一行，直接 echo 的话就解析不了了，不过 echo 可以支持”\n”转义，就是加-e看一下解释和示例，我这里使用了 tldr ，可以用 npm install -g tldr 安装，也可以直接用man， 或者–help 来查看使用方式<br><img data-src="https://img.nicksxs.me/uPic/TWcqJz.jpg"></p>
|
|
<h2 id="查看镜像底包"><a href="#查看镜像底包" class="headerlink" title="查看镜像底包"></a>查看镜像底包</h2><p>还有一点也是在这个时候要安装 vim 之类的，得知道是什么镜像底包，如果是用 uname 指令，其实看到的是宿主机的系统，得用<code>cat /etc/issue</code></p>
|
|
<p><img data-src="https://img.nicksxs.me/uPic/HFW1Xa.jpg"><br>这里稍稍记一下</p>
|
|
<h2 id="寻找系统镜像源"><a href="#寻找系统镜像源" class="headerlink" title="寻找系统镜像源"></a>寻找系统镜像源</h2><p>目前国内系统源用得比较多的是阿里云源，不过这里也推荐<a href="https://mirrors.tuna.tsinghua.edu.cn/">清华源</a>, <a href="https://mirrors.ustc.edu.cn/">中科大源</a>, <a href="https://mirrors.zju.edu.cn/">浙大源</a> 这里不要脸的推荐下母校的源，不过还不是很完善，尽情期待下。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Linux</category>
|
|
<category>Docker</category>
|
|
<category>命令</category>
|
|
<category>echo</category>
|
|
<category>发行版本</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>linux</tag>
|
|
<tag>Docker</tag>
|
|
<tag>echo</tag>
|
|
<tag>uname</tag>
|
|
<tag>发行版</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>github 小技巧-更新 github host key</title>
|
|
<url>/2023/03/28/github-%E5%B0%8F%E6%8A%80%E5%B7%A7-%E6%9B%B4%E6%96%B0-github-host-key/</url>
|
|
<content><![CDATA[<p>最近一次推送博客，发现报了个错推不上去，</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!</span><br><span class="line"></span><br><span class="line">IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!</span><br><span class="line">Someone could be eavesdropping on you right now (man-in-the-middle attack)!</span><br><span class="line">It is also possible that a host key has just been changed.</span><br></pre></td></tr></table></figure>
|
|
<p>错误信息是这样，有点奇怪也没干啥，网上一搜发现是<a href="https://github.blog/2023-03-23-we-updated-our-rsa-ssh-host-key/">We updated our RSA SSH host key</a><br>简单翻一下就是</p>
|
|
<blockquote>
|
|
<p>在3月24日协调世界时大约05:00时，出于谨慎，我们更换了用于保护 GitHub.com 的 Git 操作的 RSA SSH 主机密钥。我们这样做是为了保护我们的用户免受任何对手模仿 GitHub 或通过 SSH 窃听他们的 Git 操作的机会。此密钥不授予对 GitHub 基础设施或客户数据的访问权限。此更改仅影响通过使用 RSA 的 SSH 进行的 Git 操作。GitHub.com 和 HTTPS Git 操作的网络流量不受影响。</p>
|
|
</blockquote>
|
|
<p>要解决也比较简单就是重置下 host key，</p>
|
|
<blockquote>
|
|
<p>Host Key是服务器用来证明自己身份的一个永久性的非对称密钥</p>
|
|
</blockquote>
|
|
<p>使用 </p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">ssh-keygen -R github.com</span><br></pre></td></tr></table></figure>
|
|
<p>然后在首次建立连接的时候同意下就可以了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>ssh</category>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>ssh</tag>
|
|
<tag>端口转发</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>Leetcode 31 Next Permutation 题解分析</title>
|
|
<url>/2024/05/06/Leetcode-31-Next-Permutation-%E9%A2%98%E8%A7%A3%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<p>这题的题目介绍有一定的误导性，没有把字典序说明白，虽然的确是常规意义的字典序，但是题目介绍给的序列顺序反而会让人觉得那样不对<br>这里我们逐步深入地来讲这个题，也类似于找规律<br>注意：这里的递减递增序列都是从前往后看的<br>字典序可以就当查字典，或者默认的字符串排序方式，就是对于一个序列，逐个字符对比，同一位的相同再对比下一位的，例如 11在12前面，12在13前面，<br>那对于同一组字符或者像题中的数字，以最简单的12来分析，因为这个就两种排列，12跟21，按字典序排列，12在前，21在后<br>通过这个仔细点也能发现，或者再举个例子123，这个排列就会多一些了 </p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">123</span><br><span class="line">132</span><br><span class="line">213</span><br><span class="line">231</span><br><span class="line">312</span><br><span class="line">321</span><br></pre></td></tr></table></figure>
|
|
<p>一共有这么几种，不过举这个例子是为了更具体的发现规律，对于第一个数字相同的情况，比如123跟132，是132在后，发现规律没，递增序列是最小的，递减序列是最大的，因为2个数字 只有两种排列，扩展到3个数字，也可以把它拆分开，同样是第一个数字是1，后面两个数字就又回到了前面12序列的情况，<br>所以就可以把问题分成两部分来解决，首先从后往前找到递减序列<br>通过例子来讲 </p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">1 4 6 5 3 2</span><br></pre></td></tr></table></figure>
|
|
<p>为什么要找到递减序列，因为递减序列已经是最大的了，要做调整要在这个之前调整，并且要找的是下一个序列，所以要是最接近的那个，否则我找到的可能就是下N个了，这个有点类似于数学里进位的意思，比如十九，九已经是个位里最大的了，找下一个比它大的只能是前面一位加一，但是加一以后需要九变到这一位的最小，也就是零。<br>对于上面的例子，递减序列就是6532，第一个非递减的是4，我要找下一个就是要比当前这个大，那我就去后面找比4大的，至于为什么能找到，因为这是递减序列边界之外的，说明肯定有比它大的，那要找哪个呢，要找比4大的数字里最小的，这样才是最近的下一个序列，并且因为后面是递减序列，所以可以直接从后往前找第一个<br>也就是会找到 5<br>找到 5 之后就做交换 </p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">1 5 6 4 3 2</span><br></pre></td></tr></table></figure>
|
|
<p>这样是不是就行了呢，不行，因为这不是下一个，而是下N个，当4变成5之后，这个序列不管5后面的数字怎么变都是比 146532 大的，但是要找的是下一个，最接近的那个，则需要后面的序列变成最小，也就是变成递增序列，也就是 </p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">1 5 2 3 4 6</span><br></pre></td></tr></table></figure>
|
|
<p>这里有两种例外情况，或者严格来说是一种<br>第一种是本身整个就是递减序列了，这在题中说了，下一个就是从头再开始，那就也很简单整个排序成递增的就行了<br>第二种是本身就是个完全递增的，那其实就交换末两位，但这种跟上面的可以糅合在一起，比如就是 </p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">1 2 3 4 5 6</span><br></pre></td></tr></table></figure>
|
|
<p>单个6构不成递增或递减，找到的第一个非递减的数字就是 5 ，那就是在 5 后面找一个比它大的做交换，这时只有 6 ，所以就 5 跟 6 交换下，然后单个 5 做排序也等于不用做<br>再结合代码来看 </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">nextPermutation</span><span class="params">(<span class="type">int</span>[] nums)</span> {</span><br><span class="line"> <span class="type">int</span> <span class="variable">n</span> <span class="operator">=</span> nums.length, i = n - <span class="number">2</span>, j = n - <span class="number">1</span>;</span><br><span class="line"> <span class="comment">// 这个while循环是为了从后往前找到递减序列的分界点</span></span><br><span class="line"> <span class="comment">// 循环完成后i就是递减序列左边的第一个数字位置</span></span><br><span class="line"> <span class="keyword">while</span> (i >= <span class="number">0</span> && nums[i] >= nums[i + <span class="number">1</span>]) --i;</span><br><span class="line"> <span class="keyword">if</span> (i >= <span class="number">0</span>) {</span><br><span class="line"> <span class="comment">// 表示有找到分界点，然后就从后往前找到第一个比nums[i]大的做交换</span></span><br><span class="line"> <span class="keyword">while</span> (nums[j] <= nums[i]) --j;</span><br><span class="line"> <span class="comment">// 这个while循环在找到第一个nums[j] > nums[i]时退出</span></span><br><span class="line"> <span class="comment">// 下面就是做交换</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">temp</span> <span class="operator">=</span> nums[i];</span><br><span class="line"> nums[i] = nums[j];</span><br><span class="line"> nums[j] = temp;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 因为前面已经保证了逆序，这里只要排成正序就是最小的了</span></span><br><span class="line"> Arrays.sort(nums, i+<span class="number">1</span>, nums.length);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>java</tag>
|
|
<tag>题解</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>NX30Pro 刷成 Openwrt-ImmortalWrt 后作为有线中继的配置方法</title>
|
|
<url>/2024/06/16/NX30Pro-%E5%88%B7%E6%88%90-Openwrt-ImmortalWrt-%E5%90%8E%E4%BD%9C%E4%B8%BA%E6%9C%89%E7%BA%BF%E4%B8%AD%E7%BB%A7%E7%9A%84%E9%85%8D%E7%BD%AE%E6%96%B9%E6%B3%95/</url>
|
|
<content><![CDATA[<p>因为买了两个NX30Pro，主要是奔着可以刷Openwrt，然后把两个都刷成了ImmortalWrt，结果发现要作为有线中继的话设置还有点麻烦，所以记录下<br>首先要设置个与主路由相同网段的静态地址，方便管理<br>在网络-LAN-基本设置-协议设置成“静态地址”<br>然后IPv4地址就是刚才说的静态地址，可以设个好记的<br>子网掩码就默认的，255.255.255.0，<br>IPv4网关就是主路由地址，DNS也设置成主路由地址<br>基础设置改成忽略此接口-不提供DHCP服务，由主路由提供<br>然后在高级设置中将IPv6的 路由通告服务和DHCPv6服务跟NDP代理禁用掉，<br>物理设置中勾选桥接接口<br>目前我是这样设置可以作为有线中继连通网络了，后续有变化继续更新</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>openwrt</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>openwrt</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>gogs使用webhook部署react单页应用</title>
|
|
<url>/2020/02/22/gogs%E4%BD%BF%E7%94%A8webhook%E9%83%A8%E7%BD%B2react%E5%8D%95%E9%A1%B5%E5%BA%94%E7%94%A8/</url>
|
|
<content><![CDATA[<p>众所周知，我是个前端彩笔，但是也想做点自己可以用的工具页面，所以就让朋友推荐了蚂蚁出品的 ant design，说基本可以直接 ctrl-c ctrl-v，实测对我这种来说还是有点难的，不过也能写点，但是现在碰到的问题是怎么部署到自己的服务器上去<br>用 ant design 写的是个单页应用，实际来说就是一个 html 加 css 跟 js，最初的时候是直接 build 完就 scp 上去，也考虑过 rsync 之类的，但是都感觉不够自动化，正好自己还没这方面的经验就想折腾下，因为我自己搭的仓库应用是 gogs，搜了一下主要是跟 drones 配合做 ci/cd，研究了一下发现其实这个事情没必要这么搞(PS：drone 也不好用)，整个 hook 就可以了, 但是实际上呢，这东西也不是那么简单<br>首先是需要在服务器上装 <a href="https://github.com/adnanh/webhook">webhook</a>，这个我一开始用 snap 安装，但是出现问题，run 的时候会出现后面参数带的 hooks.json 文件找不到，然后索性就直接 github 上下最新版，放 /usr/local/bin 了，webhook 的原理呢其实也比较简单，就是起一个 http 服务，通过 post 请求调用，解析下参数，如果跟配置的参数一致，就调用对应的命令或者脚本。</p>
|
|
<h2 id="配置-hooks-json"><a href="#配置-hooks-json" class="headerlink" title="配置 hooks.json"></a>配置 hooks.json</h2><p>webhook 的配置，需要的两个文件，一个是 hooks.json，这个是 webhook 服务的配置文件，像这样</p>
|
|
<figure class="highlight json"><table><tr><td class="code"><pre><span class="line"><span class="punctuation">[</span></span><br><span class="line"> <span class="punctuation">{</span></span><br><span class="line"> <span class="attr">"id"</span><span class="punctuation">:</span> <span class="string">"redeploy-app"</span><span class="punctuation">,</span></span><br><span class="line"> <span class="attr">"execute-command"</span><span class="punctuation">:</span> <span class="string">"/opt/scripts/redeploy.sh"</span><span class="punctuation">,</span></span><br><span class="line"> <span class="attr">"command-working-directory"</span><span class="punctuation">:</span> <span class="string">"/opt/scripts"</span><span class="punctuation">,</span></span><br><span class="line"> <span class="attr">"pass-arguments-to-command"</span><span class="punctuation">:</span></span><br><span class="line"> <span class="punctuation">[</span></span><br><span class="line"> <span class="punctuation">{</span></span><br><span class="line"> <span class="attr">"source"</span><span class="punctuation">:</span> <span class="string">"payload"</span><span class="punctuation">,</span></span><br><span class="line"> <span class="attr">"name"</span><span class="punctuation">:</span> <span class="string">"head_commit.message"</span></span><br><span class="line"> <span class="punctuation">}</span><span class="punctuation">,</span></span><br><span class="line"> <span class="punctuation">{</span></span><br><span class="line"> <span class="attr">"source"</span><span class="punctuation">:</span> <span class="string">"payload"</span><span class="punctuation">,</span></span><br><span class="line"> <span class="attr">"name"</span><span class="punctuation">:</span> <span class="string">"pusher.name"</span></span><br><span class="line"> <span class="punctuation">}</span><span class="punctuation">,</span></span><br><span class="line"> <span class="punctuation">{</span></span><br><span class="line"> <span class="attr">"source"</span><span class="punctuation">:</span> <span class="string">"payload"</span><span class="punctuation">,</span></span><br><span class="line"> <span class="attr">"name"</span><span class="punctuation">:</span> <span class="string">"head_commit.id"</span></span><br><span class="line"> <span class="punctuation">}</span></span><br><span class="line"> <span class="punctuation">]</span><span class="punctuation">,</span></span><br><span class="line"> <span class="attr">"trigger-rule"</span><span class="punctuation">:</span></span><br><span class="line"> <span class="punctuation">{</span></span><br><span class="line"> <span class="attr">"and"</span><span class="punctuation">:</span></span><br><span class="line"> <span class="punctuation">[</span></span><br><span class="line"> <span class="punctuation">{</span></span><br><span class="line"> <span class="attr">"match"</span><span class="punctuation">:</span></span><br><span class="line"> <span class="punctuation">{</span></span><br><span class="line"> <span class="attr">"type"</span><span class="punctuation">:</span> <span class="string">"payload-hash-sha1"</span><span class="punctuation">,</span></span><br><span class="line"> <span class="attr">"secret"</span><span class="punctuation">:</span> <span class="string">"your-github-secret"</span><span class="punctuation">,</span></span><br><span class="line"> <span class="attr">"parameter"</span><span class="punctuation">:</span></span><br><span class="line"> <span class="punctuation">{</span></span><br><span class="line"> <span class="attr">"source"</span><span class="punctuation">:</span> <span class="string">"header"</span><span class="punctuation">,</span></span><br><span class="line"> <span class="attr">"name"</span><span class="punctuation">:</span> <span class="string">"X-Hub-Signature"</span></span><br><span class="line"> <span class="punctuation">}</span></span><br><span class="line"> <span class="punctuation">}</span></span><br><span class="line"> <span class="punctuation">}</span><span class="punctuation">,</span></span><br><span class="line"> <span class="punctuation">{</span></span><br><span class="line"> <span class="attr">"match"</span><span class="punctuation">:</span></span><br><span class="line"> <span class="punctuation">{</span></span><br><span class="line"> <span class="attr">"type"</span><span class="punctuation">:</span> <span class="string">"value"</span><span class="punctuation">,</span></span><br><span class="line"> <span class="attr">"value"</span><span class="punctuation">:</span> <span class="string">"refs/heads/master"</span><span class="punctuation">,</span></span><br><span class="line"> <span class="attr">"parameter"</span><span class="punctuation">:</span></span><br><span class="line"> <span class="punctuation">{</span></span><br><span class="line"> <span class="attr">"source"</span><span class="punctuation">:</span> <span class="string">"payload"</span><span class="punctuation">,</span></span><br><span class="line"> <span class="attr">"name"</span><span class="punctuation">:</span> <span class="string">"ref"</span></span><br><span class="line"> <span class="punctuation">}</span></span><br><span class="line"> <span class="punctuation">}</span></span><br><span class="line"> <span class="punctuation">}</span></span><br><span class="line"> <span class="punctuation">]</span></span><br><span class="line"> <span class="punctuation">}</span></span><br><span class="line"> <span class="punctuation">}</span></span><br><span class="line"><span class="punctuation">]</span></span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这是个跟 github搭配的示例，首先 id 表示的是这个对应 hook 的识别 id，也可以看到这个 hooks.json 的结构是这样的一个数组，然后就是要执行的命令和命令执行的参数，值得注意的是这个<code>trigger-rule</code>,就是请求进来了回去匹配里面的，比如前一个是一个加密的，放在请求头里，第二个 match 表示请求里的 ref 是个 master 分支，就可以区分分支进行不同操作，但是前面的加密配合 gogs 使用的时候有个问题(PS: webhook 的文档是真的烂)，gogs 设置 webhook 的加密是用的</p>
|
|
<blockquote>
|
|
<p>密钥文本将被用于计算推送内容的 SHA256 HMAC 哈希值，并设置为 <code>X-Gogs-Signature</code> 请求头的值。</p>
|
|
</blockquote>
|
|
<p>这种加密方式，所以 webhook 的这个示例的加密方式不行，但这货的文档里居然没有说明支持哪些加密，神TM，后来还是翻 <a href="https://github.com/adnanh/webhook/issues/289">issue</a> 翻到了, 需要使用这个<code>payload-hash-sha256</code></p>
|
|
<h2 id="执行脚本-redeploy-sh"><a href="#执行脚本-redeploy-sh" class="headerlink" title="执行脚本 redeploy.sh"></a>执行脚本 redeploy.sh</h2><p>脚本类似于这样</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_">#</span><span class="language-bash">!/bin/bash -e</span></span><br><span class="line"></span><br><span class="line">function cleanup {</span><br><span class="line"> echo "Error occoured"</span><br><span class="line">}</span><br><span class="line">trap cleanup ERR</span><br><span class="line"></span><br><span class="line">commit_message=$1 # head_commit.message</span><br><span class="line">pusher_name=$2 # pusher.name</span><br><span class="line">commit_id=$3 # head_commit.id</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">cd ~/do-react-example-app/</span><br><span class="line">git pull origin master</span><br><span class="line">yarn && yarn build</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>就是简单的拉代码，然后构建下，真实使用时可能不是这样，因为页面会部署在 nginx 的作用目录，还需要 rsync 过去，这部分可能还涉及到两个问题第一个是使用 rsync 还是其他的 cp，不过这个无所谓；第二个是目录权限的问题，以我的系统ubuntu 为例，默认用户是 ubuntu，nginx 部署的目录是 www，所以需要切换用户等操作，一开始是想用在shell 文件中直接写了密码，但是不知道咋传，查了下是类似于这样 <code>echo "passwd" | sudo -S cmd</code>，通过管道命令往后传，然后就是这个<code>-S</code>, 参数的解释是<code>-S, --stdin read password from standard input</code>,但是这样么也不是太安全的赶脚，又看了下还有两种方法，</p>
|
|
<ul>
|
|
<li><p>就是给root 设置一个不需要密码的命令类似于这样，</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">myusername ALL = (ALL) ALL</span><br><span class="line">myusername ALL = (root) NOPASSWD: /path/to/my/program</span><br></pre></td></tr></table></figure>
|
|
</li>
|
|
<li><p>另一种就是把默认用户跟 root 设置成同一个 group 的</p>
|
|
</li>
|
|
</ul>
|
|
<h2 id="使用"><a href="#使用" class="headerlink" title="使用"></a>使用</h2><p>真正实操的时候其实还有不少问题，首先运行 webhook 就碰到了我前面说的，使用 snap 运行的时候会找不到前面的 hooks.json配置文件，执行<code>snap run webhook -hooks /opt/hooks/hooks.json -verbose</code>就碰到下面的<code>couldn't load hooks from file! open /opt/hooks/hooks.json: no such file or directory</code>，后来直接下了个官方最新的 release，就直接执行 <code>webhook -hooks /opt/hooks/hooks.json -verbose</code> 就可以了，然后是前面的示例配置文件里的几个参数，比如<code>head_commit.message</code> 其实 gogs 推过来的根本没这玩意，而且都是数组，不知道咋取，烂文档，不过总比搭个 drone 好一点就忍了。补充一点就是在 debug 的时候需要看下问题出在哪，看看脚本有没有执行，所以需要在前面的 json 里加这个参数<code>"include-command-output-in-response": true</code>, 就能输出来脚本执行结果</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>持续集成</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Gogs</tag>
|
|
<tag>Webhook</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>git小技巧之git cherry-pick</title>
|
|
<url>/2024/11/09/git%E5%B0%8F%E6%8A%80%E5%B7%A7%E4%B9%8Bgit-cherry-pick/</url>
|
|
<content><![CDATA[<p>上次讲了 <code>git stash</code> 跟 <code>git commit</code> 的小技巧，这次来讲下另一个命令，<code>git cherry-pick</code>，我怀疑这个命令应该叫 <code>choose-pick</code> 才比较好<br>这里有个使用场景，比如我们有分支A跟B，分支B想用一下分支A一次变更的代码，正好不用自己再开发了<br>比如我在主分支里提交了一个分支B也想要使用的代码或者内容<br><img data-src="https://img.nicksxs.me/blog/TNnxi9.png"><br>然后在后面再提交一次<br><img data-src="https://img.nicksxs.me/blog/SqPYe0.png"><br>切到branchB，做一次提交<br><img data-src="https://img.nicksxs.me/blog/Lj42Z1.png"><br>我此时就想把主分支之前那个提交内容给应用到我这个branchB，那就可以用cherry-pick<br>找到master分支上那个提交 <code>commit id</code><br>用 <code>git cherry-pick {commit id}</code><br><img data-src="https://img.nicksxs.me/blog/zslvvZ.png"><br>可以看到这个commit已经被我挑过来应用上了<br><img data-src="https://img.nicksxs.me/blog/VRYp1B.png"><br>然后再深度使用还有比如是否要产生新提交，还是只在暂存区，以及如果发生冲突了是继续还是停止等等</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>git</category>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>git</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>git小技巧之查看git commit</title>
|
|
<url>/2024/11/02/git%E5%B0%8F%E6%8A%80%E5%B7%A7%E4%B9%8B%E6%9F%A5%E7%9C%8Bgit-commit/</url>
|
|
<content><![CDATA[<p>上次说的<code>git stash</code>我觉得是个非常有用的功能，简直是个大杀器，除非不存在需要切换分支的场景，否则在频繁切换的时候，<code>git stash</code> 可以让我们的commit更清晰，不至于写到一半就提交，频繁切换，甚至连<code>commit</code>信息都写不好<br>这次的小技巧是属于一个延伸知识，我们知道每次<code>commit</code>都会提交我们的变更，然后我们可以通过<code>git diff .</code>查看当前仓库产生的未提交的变更，如果我想看某一次<code>commit</code>的变更呢，这个功能通过 <code>git</code> 的各种 <code>gui</code> 工具可以很方便的查看，如果通过命令其实也很方便，我们每一次的<code>commit</code> 都会产生一个 <code>commit id</code> , 看过我上一篇的可以联想到，是否 <code>git commit</code> 也有这种 <code>show</code> 命令，答案是肯定的，<br>首先我们可以通过 <code>git log</code> 来查看我们的提交记录<br><img data-src="https://img.nicksxs.me/blog/elBcTj.png"><br>这里我们可以看到我们的提交 <code>commit id</code><br>然后通过 <code>git show xxxxx</code> 来查看我们这次 <code>commit</code> 修改的内容, <code>xxxxx</code> 表示我们的 <code>commit id</code><br><img data-src="https://img.nicksxs.me/blog/L1BW0R.png"><br>再延伸下，如果想看某个文件的变更呢，也是可以的，就是在 <code>git show xxxxx </code> 后面跟上文件名<br>比如<br><code>git show xxxxx 1.txt</code><br>就能看到这一次 <code>commit</code> 对 <code>1.txt</code> 这个文件的更改，都是比较简单入门的 <code>git</code> 命令的小延伸，只是觉得对我有点帮助就分享下</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>git</category>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>git</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>headscale 添加节点</title>
|
|
<url>/2023/07/09/headscale-%E6%B7%BB%E5%8A%A0%E8%8A%82%E7%82%B9/</url>
|
|
<content><![CDATA[<h1 id="添加节点"><a href="#添加节点" class="headerlink" title="添加节点"></a>添加节点</h1><p>添加节点非常简单，比如 app store 或者官网可以下载 mac 的安装包，</p>
|
|
<p>安装包直接下载可以在<a href="%5Bhttps://pkgs.tailscale.com/stable/#macos%5D(https://pkgs.tailscale.com/stable/#macos)">这里</a>，下载安装完后还需要做一些处理，才能让 Tailscale 使用 Headscale 作为控制服务器。当然，Headscale 已经给我们提供了详细的操作步骤，你只需要在浏览器中打开 URL：<code>http://<HEADSCALE_PUB_IP>:<HEADSCALE_PUB_PORT>/apple</code>，记得端口替换成自己的，就会看到这样的说明页</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/FzL6Vc.png" alt="image"></p>
|
|
<p>然后对于像我这样自己下载的客户端安装包，也就是standalone client，就可以用下面的命令</p>
|
|
<p><code>defaults write io.tailscale.ipn.macsys ControlURL http://<HEADSCALE_PUB_IP>:<HEADSCALE_PUB_PORT> </code> 类似于 Windows 客户端需要写入注册表，就是把控制端的地址改成了我们自己搭建的 headscale 的，设置完以后就打开 tailscale 客户端右键点击 login，就会弹出一个浏览器地址</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/Gy3oxP.jpg" alt="image"></p>
|
|
<p>按照这个里面的命令去 headscale 的机器上执行，注意要替换 namespace，对于最新的 headscale 已经把 namespace 废弃改成 user 了，这点要注意了，其他客户端也同理，现在还有个好消息，安卓和 iOS 客户端也已经都可以用了，后面可以在介绍下局域网怎么部分打通和自建 derper。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>headscale</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>headscale</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>hexo 配置系列-接入Algolia搜索</title>
|
|
<url>/2023/04/02/hexo-%E9%85%8D%E7%BD%AE%E7%B3%BB%E5%88%97-%E6%8E%A5%E5%85%A5Algolia%E6%90%9C%E7%B4%A2/</url>
|
|
<content><![CDATA[<p>博客之前使用的是 local search，最开始感觉使用体验还不错，速度也不慢，最近自己搜了下觉得效果差了很多，不知道是啥原因，所以接入有 next 主题支持的 Algolia 搜索，next 主题的文档已经介绍的很清楚了，这边就记录下，<br>首先要去 Algolia 开通下账户，创建一个索引<br><img data-src="https://img.nicksxs.me/blog/RRzIVK.png"><br>创建好后要去找一下 api key 的配置，这个跟 next 主题的说明已经有些不一样了<br>在设置里可以找到<br><img data-src="https://img.nicksxs.me/blog/s9zsVv.png"><br>这里默认会有两个 key<br><img data-src="https://img.nicksxs.me/blog/Fg6YzL.png"><br>一个是 search only,一个是 admin key，需要再创建一个自定义 key<br>这个 key 需要有这些权限，称为 <code>High-privilege API key</code>， 后面有用<br><img data-src="https://img.nicksxs.me/blog/2d9vAa.png"><br>然后就是到博客目录下安装</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">cd hexo-site</span><br><span class="line">npm install hexo-algolia</span><br></pre></td></tr></table></figure>
|
|
<p>然后在 hexo 站点配置中添加</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">algolia:</span><br><span class="line"> applicationID: "Application ID"</span><br><span class="line"> apiKey: "Search-only API key"</span><br><span class="line"> indexName: "indexName"</span><br></pre></td></tr></table></figure>
|
|
<p>包括应用 Id，只搜索的 api key（默认给创建好的那个）,indexName 就是最开始创建的 index 名，</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">export HEXO_ALGOLIA_INDEXING_KEY=High-privilege API key # Use Git Bash</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash"><span class="built_in">set</span> HEXO_ALGOLIA_INDEXING_KEY=High-privilege API key <span class="comment"># Use Windows command line</span></span></span><br><span class="line">hexo clean</span><br><span class="line">hexo algolia</span><br></pre></td></tr></table></figure>
|
|
<p>然后再到 next 配置中开启 <code>algolia_search</code></p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_"># </span><span class="language-bash">Algolia Search</span></span><br><span class="line">algolia_search:</span><br><span class="line"> enable: true</span><br><span class="line"> hits:</span><br><span class="line"> per_page: 10</span><br></pre></td></tr></table></figure>
|
|
<p>搜索的界面其实跟 local 的差不多，就是搜索效果会好一些<br><img data-src="https://img.nicksxs.me/blog/u06Pux.png"><br>也推荐可以搜搜过往的内容，已经左边有个热度的，做了个按阅读量排序的榜单。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>hexo</category>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>hexo</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>invert-binary-tree</title>
|
|
<url>/2015/06/22/invert-binary-tree/</url>
|
|
<content><![CDATA[<p><strong>Invert a binary tree</strong></p>
|
|
<pre><code> 4
|
|
/ \
|
|
2 7
|
|
/ \ / \
|
|
1 3 6 9
|
|
</code></pre>
|
|
<p>to</p>
|
|
<pre><code> 4
|
|
/ \
|
|
7 2
|
|
/ \ / \
|
|
9 6 3 1
|
|
</code></pre>
|
|
<p><strong>Trivia:</strong><br>This problem was inspired by <a href="https://twitter.com/mxcl/status/608682016205344768">this original tweet</a> by <a href="https://twitter.com/mxcl">Max Howell</a>:</p>
|
|
<blockquote>
|
|
<p>Google: 90% of our engineers use the software you wrote (Homebrew),<br>but you can’t invert a binary tree on a whiteboard so fuck off. </p>
|
|
</blockquote>
|
|
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Definition for a binary tree node.</span></span><br><span class="line"><span class="comment"> * struct TreeNode {</span></span><br><span class="line"><span class="comment"> * int val;</span></span><br><span class="line"><span class="comment"> * TreeNode *left;</span></span><br><span class="line"><span class="comment"> * TreeNode *right;</span></span><br><span class="line"><span class="comment"> * TreeNode(int x) : val(x), left(NULL), right(NULL) {}</span></span><br><span class="line"><span class="comment"> * };</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">class</span> <span class="title class_">Solution</span> {</span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line"> <span class="function">TreeNode* <span class="title">invertTree</span><span class="params">(TreeNode* root)</span> </span>{</span><br><span class="line"> <span class="keyword">if</span>(root == <span class="literal">NULL</span>) <span class="keyword">return</span> root;</span><br><span class="line"> TreeNode* temp;</span><br><span class="line"> temp = <span class="built_in">invertTree</span>(root->left);</span><br><span class="line"> root->left = <span class="built_in">invertTree</span>(root->right);</span><br><span class="line"> root->right = temp;</span><br><span class="line"> <span class="keyword">return</span> root;</span><br><span class="line"> }</span><br><span class="line">};</span><br></pre></td></tr></table></figure>]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>c++</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>java 中发起 http 请求时证书问题解决记录</title>
|
|
<url>/2023/07/29/java-%E4%B8%AD%E5%8F%91%E8%B5%B7-http-%E8%AF%B7%E6%B1%82%E6%97%B6%E8%AF%81%E4%B9%A6%E9%97%AE%E9%A2%98%E8%A7%A3%E5%86%B3%E8%AE%B0%E5%BD%95/</url>
|
|
<content><![CDATA[<p>再一次环境部署是发现了个问题，就是在请求微信 https 请求的时候，出现了个错误<br><code>No appropriate protocol (protocol is disabled or cipher suites are inappropriate)</code><br>一开始以为是环境问题，从 oracle 的 jdk 换成了基于 openjdk 的底包，没有 javax 的关系，<br>完整的提示包含了 javax 的异常<br><code>java.lang.RuntimeException: javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)</code><br>后面再看了下，是不是也可能是证书的问题，然后就去找了下是不是证书相关的，<br>可以看到在 <code>/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/security</code> 路径下的 <code>java.security</code> 中<br><code>jdk.tls.disabledAlgorithms=SSLv3, TLSv1, TLSv1.1, RC4, DES, MD5withRSA, </code><br>而正好在我们代码里 <code>createSocketFactory</code> 的时候使用了 TLSv1 这个证书协议</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">SSLContext</span> <span class="variable">sslContext</span> <span class="operator">=</span> SSLContext.getInstance(<span class="string">"TLS"</span>);</span><br><span class="line">sslContext.init(kmf.getKeyManagers(), <span class="literal">null</span>, <span class="keyword">new</span> <span class="title class_">SecureRandom</span>());</span><br><span class="line"><span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">SSLConnectionSocketFactory</span>(sslContext, <span class="keyword">new</span> <span class="title class_">String</span>[]{<span class="string">"TLSv1"</span>}, <span class="literal">null</span>, <span class="keyword">new</span> <span class="title class_">DefaultHostnameVerifier</span>());</span><br></pre></td></tr></table></figure>
|
|
<p>所以就有两种方案，一个是使用更新版本的 TLS 或者另一个就是使用比较久的 jdk，这也说明其实即使都是 jdk8 的，不同的小版本差异还是会有些影响，有的时候对于这些错误还是需要更深入地学习，不能一概而之认为就是 jdk 用的是 oracle 还是 openjdk 的，不同的错误可能就需要仔细确认原因所在。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>git小技巧之git stash</title>
|
|
<url>/2024/10/27/git%E5%B0%8F%E6%8A%80%E5%B7%A7%E4%B9%8Bgit-stash/</url>
|
|
<content><![CDATA[<p>我们日常开发包括我自用的小工具也在用git管理<br>在使用git的过程中经常有个场景是我在A分支上开发了一部分，临时需要切换到B分支，又不想先把这部分代码提交，因为还没开发完，<br>这是用<code>git stash</code>命令就能很好的解决这个问题，但是这里在使用<code>git stash</code>的时候我认为有两个阶段<br>第一阶段就是简单使用<br><img data-src="https://img.nicksxs.me/blog/8Devzx.png"><br><img data-src="https://img.nicksxs.me/blog/Q68J3B.png"><br><img data-src="https://img.nicksxs.me/blog/JtUJ1J.png"><br>比如我有这个进行中的变更，在有未提交的代码的时候可以先用</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">git add 1.txt</span><br></pre></td></tr></table></figure>
|
|
<p>添加到暂存区，但不提交，然后使用</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">git stash</span><br></pre></td></tr></table></figure>
|
|
<p>保存工作进度，<code>git stash</code>之后再用 <code>git status</code> 就看不到刚才的更改了<br><img data-src="https://img.nicksxs.me/blog/kB4WT0.png"><br>然后就是弹出刚才stash的变更了<br>可以使用 <code>git stash pop</code> 就可以把刚才藏起来的临时变更就弹出来了<br><img data-src="https://img.nicksxs.me/blog/XEznjC.png"><br>这里其实在真正使用时我们如果是深度使用就可能会出现四个问题或者是诉求</p>
|
|
<h3 id="问题一"><a href="#问题一" class="headerlink" title="问题一"></a>问题一</h3><p>我不止stash了一次变更，怎么找到stash的多少次呢，因为直接用 <code>pop</code> 就是把栈顶的那次隐藏的变更弹出来了<br>这个问题就可以通过 <code>git stash list</code> 来解决<br><img data-src="https://img.nicksxs.me/blog/JSe0QQ.png"></p>
|
|
<h3 id="问题二"><a href="#问题二" class="headerlink" title="问题二"></a>问题二</h3><p>如果使用的pop我们这个stash相当于栈顶弹出，已经pop出来的就没有了，如果我只是想看一下改了哪些，当然我们可以先pop出来，在stash回去，<br>但更好的办法就是用<code>git stash apply</code>，这样是不会清栈的</p>
|
|
<h3 id="问题三"><a href="#问题三" class="headerlink" title="问题三"></a>问题三</h3><p>如何指定弹出，比如我在list里能看到多次stash，那么我想弹出其中某一次<br>可以用<br><code>git stash pop stash@{1}</code> 注意这个 stash@{1} 是指<code>stash list</code> 的第二个，因为是从0开始的<br>弹出倒数第二次<code>stash</code>的</p>
|
|
<h4 id="问题四"><a href="#问题四" class="headerlink" title="问题四"></a>问题四</h4><p>我想知道我这个stash里内容是啥，在不用pop和apply的情况下有没有办法<br>也是有的<br>可以使用 <code>git stash show 1</code> 或者 <code>git stash show stash@{1}</code> 来查看这次变更了哪些文件<br>甚至我们可以用 <code>git stash show 1 -p</code> 来查看具体变更的内容</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>git</category>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>git</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>grep小技巧之匹配到二进制文件</title>
|
|
<url>/2024/11/24/grep%E5%B0%8F%E6%8A%80%E5%B7%A7%E4%B9%8B%E5%8C%B9%E9%85%8D%E5%88%B0%E4%BA%8C%E8%BF%9B%E5%88%B6%E6%96%87%E4%BB%B6/</url>
|
|
<content><![CDATA[<p>我们在日常使用grep这个强大的命令行工具时有时候会碰到一个问题，就是grep在识别文件的时候在碰到 ‘\000 NUL’的时候会认为文件是二进制文件，就不进行识别了，碰到这种情况我们可以使用 <code>grep -a</code> 或者 <code>grep --text</code><br>比如我们做个小实验</p>
|
|
<figure class="highlight php"><table><tr><td class="code"><pre><span class="line"><span class="meta"><?php</span></span><br><span class="line"><span class="variable">$content</span> = <span class="title function_ invoke__">file_get_contents</span>(<span class="string">'input.txt'</span>);</span><br><span class="line"><span class="variable">$binary</span> = <span class="string">"\0"</span> . <span class="variable">$content</span>; <span class="comment">// 添加 NUL 字符到开头</span></span><br><span class="line"><span class="title function_ invoke__">file_put_contents</span>(<span class="string">'output.bin'</span>, <span class="variable">$binary</span>);</span><br></pre></td></tr></table></figure>
|
|
<p>然后我们就有了 <code>output.bin</code> 这个二进制文件<br>本身 <code>input.txt</code> 的内容也很简单</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">12345</span><br><span class="line">12345</span><br></pre></td></tr></table></figure>
|
|
<p>我们尝试用 <code>grep</code> 查找其中的内容，<br><img data-src="https://img.nicksxs.me/blog/reaAER.png"><br>就会出现这个问题<br>当我们加上了 <code>-a</code> 或者 <code>--text</code> 就是指定 <code>grep</code> 需要在包含二进制文件格式中或者将文件当做是文本文件来识别匹配</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>grep</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>grep命令小技巧-统计行数</title>
|
|
<url>/2024/12/15/grep%E5%91%BD%E4%BB%A4%E5%B0%8F%E6%8A%80%E5%B7%A7-%E7%BB%9F%E8%AE%A1%E8%A1%8C%E6%95%B0/</url>
|
|
<content><![CDATA[<p>之前在使用grep来匹配内容，想看下匹配的结果数量都是用管道，然后用wc命令的<br>最近看了下似乎没必要这么麻烦<br>grep命令自带了这个功能<br>原来在用grep的时候还有个额外的功能就是看匹配行的前后几行，<br>使用</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">grep -B 4</span><br></pre></td></tr></table></figure>
|
|
<p>是匹配行的前4行，可以用before来记</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">grep -A 4</span><br></pre></td></tr></table></figure>
|
|
<p>是匹配行的后4行，可以用after来记<br>然后</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">grep -C 4</span><br></pre></td></tr></table></figure>
|
|
<p>是看匹配行的前后4行<br>而这次的主角是小写的c</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">grep -c 'something' filename.txt</span><br></pre></td></tr></table></figure>
|
|
|
|
<p><img data-src="https://img.nicksxs.me/blog/lU6ari.png"><br>类似于这个效果，这又是个可以少打几个字符的偷懒技巧</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>grep</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>java的agent初体验</title>
|
|
<url>/2024/12/22/java%E7%9A%84agent%E5%88%9D%E4%BD%93%E9%AA%8C/</url>
|
|
<content><![CDATA[<p>之前在用到arthas就想到过可以研究下java的agent，这里算是个初入门<br>首先我们有个应用，需要挂上agent来探测一些事情<br>比如就是简单的主方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">Main</span> {</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> System.out.println(<span class="string">"Hello world!"</span>);</span><br><span class="line"> Demo.start();</span><br><span class="line"> Demo.start2();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后有个Demo类</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">Demo</span> {</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">start</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> System.out.println(<span class="string">"Demo starting ..."</span>);</span><br><span class="line"> Thread.sleep(<span class="number">123</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException ignore) {</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">start2</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> System.out.println(<span class="string">"Demo starting 2..."</span>);</span><br><span class="line"> Thread.sleep(<span class="number">567</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException ignore) {</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后我们想要像切面一样在start方法前后织入一些前后置的逻辑<br>我们需要将一个maven的module<br>然后agent代码是这样, premain 方法会在我们指定挂载agent执行，并且就是入口方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">DemoAgent</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">premain</span><span class="params">(String arg, Instrumentation instrumentation)</span> {</span><br><span class="line"> System.out.println(<span class="string">"agent start"</span>);</span><br><span class="line"> System.out.println(<span class="string">"arg is "</span> + arg);</span><br><span class="line"> instrumentation.addTransformer(<span class="keyword">new</span> <span class="title class_">DemoTransformer</span>());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后这个 transformer 就是我要怎么在目标类里如何织入代码的逻辑</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">DemoTransformer</span> <span class="keyword">implements</span> <span class="title class_">ClassFileTransformer</span> {</span><br><span class="line"> <span class="comment">// 这里是我们要织入代码的目标类</span></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">INJECTED_CLASS</span> <span class="operator">=</span> <span class="string">"org.example.Demo"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="type">byte</span>[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, <span class="type">byte</span>[] classfileBuffer) <span class="keyword">throws</span> IllegalClassFormatException {</span><br><span class="line"> <span class="type">String</span> <span class="variable">realClassName</span> <span class="operator">=</span> className.replace(<span class="string">"/"</span>, <span class="string">"."</span>);</span><br><span class="line"> <span class="comment">// 匹配到目标类</span></span><br><span class="line"> <span class="keyword">if</span> (realClassName.equals(INJECTED_CLASS)) {</span><br><span class="line"> System.out.println(<span class="string">"injected class ："</span> + realClassName);</span><br><span class="line"> CtClass ctClass;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 使用javassist,获取字节码类</span></span><br><span class="line"> <span class="type">ClassPool</span> <span class="variable">classPool</span> <span class="operator">=</span> ClassPool.getDefault();</span><br><span class="line"> ctClass = classPool.get(realClassName);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 获取目标类所有的方法，也可指定方法，进行增强</span></span><br><span class="line"> CtMethod[] declaredMethods = ctClass.getDeclaredMethods();</span><br><span class="line"> <span class="keyword">for</span> (CtMethod declaredMethod : declaredMethods) {</span><br><span class="line"> System.out.println(declaredMethod.getName() + <span class="string">"method be inject"</span>);</span><br><span class="line"> declaredMethod.addLocalVariable(<span class="string">"time"</span>, CtClass.longType);</span><br><span class="line"> declaredMethod.insertBefore(<span class="string">"System.out.println(\"--- start ---\");"</span>);</span><br><span class="line"> declaredMethod.insertBefore(<span class="string">"time = System.currentTimeMillis();"</span>);</span><br><span class="line"> declaredMethod.insertAfter(<span class="string">"System.out.println(\"--- end ---\");"</span>);</span><br><span class="line"> declaredMethod.insertAfter(<span class="string">"System.out.println(\"cost: \" + (System.currentTimeMillis() - time));"</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> ctClass.toBytecode();</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable ignore) {</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> classfileBuffer;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>接下去要在pom.xml里配置如何打包agent的命令</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">build</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">plugins</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.apache.maven.plugins<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>maven-compiler-plugin<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>3.5.1<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="comment"><!-- 指定maven编译的jdk版本。若不指定,maven3默认用jdk 1.5 maven2默认用jdk1.3 --></span></span><br><span class="line"> <span class="tag"><<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">source</span>></span>8<span class="tag"></<span class="name">source</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">target</span>></span>8<span class="tag"></<span class="name">target</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">plugin</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="tag"><<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.apache.maven.plugins<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>maven-jar-plugin<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>3.2.0<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">archive</span>></span></span><br><span class="line"> <span class="comment"><!--自动添加META-INF/MANIFEST.MF --></span></span><br><span class="line"> <span class="tag"><<span class="name">manifest</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">addClasspath</span>></span>true<span class="tag"></<span class="name">addClasspath</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">manifest</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">manifestEntries</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">Menifest-Version</span>></span>1.0<span class="tag"></<span class="name">Menifest-Version</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">Premain-Class</span>></span>org.example.DemoAgent<span class="tag"></<span class="name">Premain-Class</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">Can-Redefine-Classes</span>></span>true<span class="tag"></<span class="name">Can-Redefine-Classes</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">Can-Retransform-Classes</span>></span>true<span class="tag"></<span class="name">Can-Retransform-Classes</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">manifestEntries</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">archive</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">plugins</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">build</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>需要指定 Premain-Class 来指定入口方法所在的类<br>然后用 <code>mvn package</code> 进行打包<br>打包后在我们的目标代码的启动vm参数里加上</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">-javaagent:/{path}/demo-agent-1.0-SNAPSHOT.jar</span><br></pre></td></tr></table></figure>
|
|
<p>{path} 需要替换成agent的jar包所在路径<br>然后执行目标main方法<br><img data-src="https://img.nicksxs.me/blog/AGP3Q1.png"><br>就可以看到结果了，这样我们第一个agent的demo就跑起来了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>java小技巧之获取调用来源</title>
|
|
<url>/2024/11/16/java%E5%B0%8F%E6%8A%80%E5%B7%A7%E4%B9%8B%E8%8E%B7%E5%8F%96%E8%B0%83%E7%94%A8%E6%9D%A5%E6%BA%90/</url>
|
|
<content><![CDATA[<p>这个话题应该首先明确，这不是类似于 <code>open tracing</code> 的应用间调用路径，而是应用内部，举个例子，我有一个底层方法，比如DAO层的一个方法，有很多调用方，那么有什么方法，第一种是最简单，但是比较麻烦，我去每个调用方那打个日志，然后分析下日志，这个办法肯定是没错的，只是如果我的调用方很多，可能需要加日志的地方就很多了，会比较麻烦<br>第二种是我可以在这个被调用的DAO层方法里加个日志，但是要怎么获取调用方呢，这是我们可以用代码堆栈trace来实现，这个思路可以从异常的角度出发，我们在出现异常的时候，如何定位异常的代码，就是通过异常堆栈，那么堆栈是不是只有在抛异常的时候才有呢，答案是否定的，了解一点java虚拟机的应该知道我们在内存中有一个区域放的就是我们的代码调用栈，用来保存我们目前程序的调用层次逻辑以及数据，之前在<code>php</code>中也讲到过这个，那时候也是帮助理清调用逻辑和调用来源，<br>在java中如何使用这个功能呢，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">StackTrace</span> {</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> method1();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">method1</span><span class="params">()</span> {</span><br><span class="line"> StackTraceElement[] stackTrace = Thread.currentThread().getStackTrace();</span><br><span class="line"> <span class="keyword">for</span> (StackTraceElement stackTraceElement : stackTrace) {</span><br><span class="line"> System.out.println(stackTraceElement);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这个方法其实也比较简单</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> StackTraceElement[] getStackTrace() {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span> != Thread.currentThread()) {</span><br><span class="line"> <span class="comment">// check for getStackTrace permission</span></span><br><span class="line"> <span class="type">SecurityManager</span> <span class="variable">security</span> <span class="operator">=</span> System.getSecurityManager();</span><br><span class="line"> <span class="keyword">if</span> (security != <span class="literal">null</span>) {</span><br><span class="line"> security.checkPermission(</span><br><span class="line"> SecurityConstants.GET_STACK_TRACE_PERMISSION);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// optimization so we do not call into the vm for threads that</span></span><br><span class="line"> <span class="comment">// have not yet started or have terminated</span></span><br><span class="line"> <span class="keyword">if</span> (!isAlive()) {</span><br><span class="line"> <span class="keyword">return</span> EMPTY_STACK_TRACE;</span><br><span class="line"> }</span><br><span class="line"> StackTraceElement[][] stackTraceArray = dumpThreads(<span class="keyword">new</span> <span class="title class_">Thread</span>[] {<span class="built_in">this</span>});</span><br><span class="line"> StackTraceElement[] stackTrace = stackTraceArray[<span class="number">0</span>];</span><br><span class="line"> <span class="comment">// a thread that was alive during the previous isAlive call may have</span></span><br><span class="line"> <span class="comment">// since terminated, therefore not having a stacktrace.</span></span><br><span class="line"> <span class="keyword">if</span> (stackTrace == <span class="literal">null</span>) {</span><br><span class="line"> stackTrace = EMPTY_STACK_TRACE;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> stackTrace;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// Don't need JVM help for current thread</span></span><br><span class="line"> <span class="keyword">return</span> (<span class="keyword">new</span> <span class="title class_">Exception</span>()).getStackTrace();</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里面返回的 <code>StackTraceElement</code> 的内容</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">final</span> <span class="keyword">class</span> <span class="title class_">StackTraceElement</span> <span class="keyword">implements</span> <span class="title class_">java</span>.io.Serializable {</span><br><span class="line"> <span class="comment">// Normally initialized by VM (public constructor added in 1.5)</span></span><br><span class="line"> <span class="keyword">private</span> String declaringClass;</span><br><span class="line"> <span class="keyword">private</span> String methodName;</span><br><span class="line"> <span class="keyword">private</span> String fileName;</span><br><span class="line"> <span class="keyword">private</span> <span class="type">int</span> lineNumber;</span><br></pre></td></tr></table></figure>
|
|
<p>分别是定义的类，方法名，文件名，以及代码行数<br><img data-src="https://img.nicksxs.me/blog/p0p1bm.png"><br>我们就能看到这样的结果</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">java.lang.Thread.getStackTrace(Thread.java:<span class="number">1564</span>)</span><br><span class="line">StackTrace.StackTrace.method1(StackTrace.java:<span class="number">14</span>)</span><br><span class="line">StackTrace.StackTrace.main(StackTrace.java:<span class="number">11</span>)</span><br></pre></td></tr></table></figure>
|
|
<p>因为是栈，所以就是最上面的是最近的方法，而第二行其实是我的目标方法，就是前面描述的逻辑其实应该找到第3个元素，因为第1个是调用 <code>getStackTrace</code> 方法，<br>第2个是目标方法，第3个才是我们想要找的调用方方法，以及所在的行数</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>java中arraylist的sort方法的 ConcurrentModificationException 问题</title>
|
|
<url>/2025/05/18/java%E4%B8%ADarraylist%E7%9A%84sort%E6%96%B9%E6%B3%95%E7%9A%84%E9%97%AE%E9%A2%98/</url>
|
|
<content><![CDATA[<p>最近写代码的时候碰到个问题，<br>比如我们有一个对象列表，对象中有个排序字段，比如时间或者索性就有个序号</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">Demo</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="type">int</span> <span class="variable">order</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="type">int</span> <span class="title function_">getOrder</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> order;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">setOrder</span><span class="params">(<span class="type">int</span> order)</span> {</span><br><span class="line"> <span class="built_in">this</span>.order = order;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>比如这样，有一个前提，就是这个列表其实是有序的，只是为了防止万一<br>所以对这个列表进行了排序，就直接使用了ArrayList自带的sort方法，由于是不小心用在了多线程环境<br>就有出现了一个并发变更报错<br>也就是这个异常 <code>java.util.ConcurrentModificationException#ConcurrentModificationException()</code><br>看下注释</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">This exception may be thrown by methods that have detected concurrent modification of an object when such modification is not permissible.</span><br><span class="line">For example, it is not generally permissible <span class="keyword">for</span> one thread to modify a Collection <span class="keyword">while</span> another thread is iterating over it. In general, the results of the iteration are undefined under these circumstances. Some Iterator <span class="title function_">implementations</span> <span class="params">(including those of all the general purpose collection implementations provided by the JRE)</span> may choose to <span class="keyword">throw</span> <span class="built_in">this</span> exception <span class="keyword">if</span> <span class="built_in">this</span> behavior is detected. Iterators that <span class="keyword">do</span> <span class="built_in">this</span> are known as fail-fast iterators, as they fail quickly and cleanly, rather that risking arbitrary, non-deterministic behavior at an undetermined time in the future.</span><br></pre></td></tr></table></figure>
|
|
<p>一般情况就是并发环境下进行了变更，<br>只是这里比较特殊，这个列表没有进行增删，同时是有序的，理论上应该是不会出现变更才对<br>但是它的确出现了，就得看下这个产生的原因<br>主要还是来看下这个<code>java.util.ArrayList#sort</code>的实现跟这个异常的抛出原理</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">sort</span><span class="params">(Comparator<? <span class="built_in">super</span> E> c)</span> {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">expectedModCount</span> <span class="operator">=</span> modCount;</span><br><span class="line"> Arrays.sort((E[]) elementData, <span class="number">0</span>, size, c);</span><br><span class="line"> <span class="keyword">if</span> (modCount != expectedModCount)</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">ConcurrentModificationException</span>();</span><br><span class="line"> modCount++;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>可以看到在排序前会有个 <code>modCount</code> 的赋值，如果在进行排序后，这个初始的 <code>modCount</code> 跟当前的不等的话就会抛出这个异常<br>那么什么时候会出现这两者不相等呢<br>简单推演下这个并发排序下的情况，线程1和线程2同时进入这个方法，拿到了一样的 <code>modCount</code> , 然后进行排序，第一个线程比较快，sort玩之后发现两者相等<br>就到了 <code>modCount++</code> , 这时候第二个线程刚排序完，发现 <code>modCount</code> 已经比 <code>expectedModCount</code> 大了<br>这个时候就出现了这个异常的抛出时机<br>如果这么理解这个异常的确是应该会出现<br>而不是我们理解的，如果不变更元素，不调整顺序，就不会触发这个并发变更的异常<br>还是有很多值得学习的点</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>java的字节码工具-javassist体验三</title>
|
|
<url>/2025/01/19/java%E7%9A%84%E5%AD%97%E8%8A%82%E7%A0%81%E5%B7%A5%E5%85%B7-javassist%E4%BD%93%E9%AA%8C%E4%B8%89/</url>
|
|
<content><![CDATA[<p>这篇还是javassist的一些使用小技巧，我们可以用javassist来读取java的注解信息<br>首先我们有这样一个注解</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="meta">@interface</span> Author {</span><br><span class="line"> String <span class="title function_">name</span><span class="params">()</span>;</span><br><span class="line"> <span class="type">int</span> <span class="title function_">year</span><span class="params">()</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>注解可以打在类上,那么我们建一个point类</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Author(name = "nick", year = 2024)</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">Point</span> {</span><br><span class="line"> <span class="type">int</span> x, y;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>name 是 nick，24年的，</p>
|
|
<p>然后我想通过javassist来获取这个类上的注解信息<br>我们先从对象池中获取这个类</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">CtClass</span> <span class="variable">cc</span> <span class="operator">=</span> ClassPool.getDefault().get(<span class="string">"org.example.Point"</span>);</span><br></pre></td></tr></table></figure>
|
|
<p>然后获取类上的注解</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">Object[] all = cc.getAnnotations();</span><br><span class="line"><span class="type">Author</span> <span class="variable">a</span> <span class="operator">=</span> (Author)all[<span class="number">0</span>];</span><br></pre></td></tr></table></figure>
|
|
<p>将第一个注解强转成Author，因为这里我们知道是只有这个注解，否则可以做的更通用一点<br>然后把这个主机上的值获取出来</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">String</span> <span class="variable">name</span> <span class="operator">=</span> a.name();</span><br><span class="line"><span class="type">int</span> <span class="variable">year</span> <span class="operator">=</span> a.year();</span><br><span class="line">System.out.println(<span class="string">"name: "</span> + name + <span class="string">", year: "</span> + year);</span><br></pre></td></tr></table></figure>
|
|
<p>可以获得结果，非常简单</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">name: nick, year: <span class="number">2024</span></span><br><span class="line"></span><br><span class="line">Process finished with exit code <span class="number">0</span></span><br></pre></td></tr></table></figure>
|
|
<p>为什么讲这么个简单的例子呢，因为这个刚好可以作为一种实现aop的途径，我们可以获取类的信息，包括注解上的，这样我们就可以从源头去理解这些aop啥的可以通过什么方法来实现</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>java小知识之String.format中的%秘密</title>
|
|
<url>/2025/03/16/java%E5%B0%8F%E7%9F%A5%E8%AF%86%E4%B9%8BString-format%E4%B8%AD%E7%9A%84-%E7%A7%98%E5%AF%86/</url>
|
|
<content><![CDATA[<p>String.format是Java中String类非常常用的一个方法，可以帮我们将占位符替换成变量，比如<code>%d</code>可以作为整型的占位符，<code>%s</code>可以作为字符串的占位符，但是吧有的时候常用归常用，有的时候碰到问题了还是得学习记录下<br>比如我们要写一个字符串替换方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">Demo</span> {</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> System.out.printf(<span class="string">"成功是%d%的努力和%d%的天赋"</span>, <span class="number">99</span>, <span class="number">1</span>);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>就是希望输出 “成功是99%的努力和1%的天赋” ，先不管这句话的正确性，看看能不能正常输出（肯定是不能）</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">Exception in thread <span class="string">"main"</span> java.util.UnknownFormatConversionException: Conversion = <span class="string">'的'</span></span><br><span class="line"> at java.util.Formatter.checkText(Formatter.java:<span class="number">2579</span>)</span><br><span class="line"> at java.util.Formatter.parse(Formatter.java:<span class="number">2555</span>)</span><br><span class="line"> at java.util.Formatter.format(Formatter.java:<span class="number">2501</span>)</span><br><span class="line"> at java.io.PrintStream.format(PrintStream.java:<span class="number">970</span>)</span><br><span class="line"> at java.io.PrintStream.printf(PrintStream.java:<span class="number">871</span>)</span><br><span class="line"> at demo.Demo.main(Demo.java:<span class="number">9</span>)</span><br></pre></td></tr></table></figure>
|
|
<p>会有这样的报错，虽然是个小问题我们还是来仔细研究下，<br>首先这边format函数会根据可能的占位符去提取并分割字符串</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">formatSpecifier</span></span><br><span class="line"> <span class="operator">=</span> <span class="string">"%(\\d+\\$)?([-#+ 0,(\\<]*)?(\\d+)?(\\.\\d+)?([tT])?([a-zA-Z%])"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="type">Pattern</span> <span class="variable">fsPattern</span> <span class="operator">=</span> Pattern.compile(formatSpecifier);</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Finds format specifiers in the format string.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">private</span> FormatString[] parse(String s) {</span><br><span class="line"> ArrayList<FormatString> al = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"> <span class="type">Matcher</span> <span class="variable">m</span> <span class="operator">=</span> fsPattern.matcher(s);</span><br></pre></td></tr></table></figure>
|
|
<p>比如上面的字符串，可以提取到两个有效的 <code>%d</code><br>然后对剩余的进行分割，分割后再遍历校验每一个字符<br>如果碰到单独的 <code>%</code> 就说明出现了错误的分隔符<br>会调用 <code>checkText</code> 来识别报错<br>而如果我们想要能够正常输出<code>%</code> 也非常简单，只需要使用两个百分号 <code>%%</code><br>这里其实就是匹配到了上面 <code>formatSpecifier</code> 的最后面，<br><code>%</code>后面可以匹配的可以使 <code>a-z</code> 和 <code>A-Z</code> 以及 <code>%</code><br>当是连续的两个 <code>%%</code> 时就会被识别成转义成实际的 <code>%</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> System.out.printf(<span class="string">"成功是%d%%的努力和%d%%的天赋"</span>, <span class="number">99</span>, <span class="number">1</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>换成这样就能正常输出了 </p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>mac os 14.x 出现 'xxx 已损坏，无法打开。 你应该将它移到废纸篓。' 解决方法</title>
|
|
<url>/2025/02/16/mac-os-14-x-%E5%87%BA%E7%8E%B0-xxx-%E5%B7%B2%E6%8D%9F%E5%9D%8F%EF%BC%8C%E6%97%A0%E6%B3%95%E6%89%93%E5%BC%80%E3%80%82-%E4%BD%A0%E5%BA%94%E8%AF%A5%E5%B0%86%E5%AE%83%E7%A7%BB%E5%88%B0%E5%BA%9F%E7%BA%B8%E7%AF%93%E3%80%82-%E8%A7%A3%E5%86%B3%E6%96%B9%E6%B3%95/</url>
|
|
<content><![CDATA[<p>在运行一个python写的小工具的时候碰到了这个问题<br>““Python.framework”已损坏，无法打开。 你应该将它移到废纸篓。”<br>一开始以为是常规问题，但是可能随着系统更新，也出现了比较不同的情况<br>首先进去”系统设置”->”隐私和安全性”<br>一开始是长这样的<br><img data-src="https://img.nicksxs.me/blog/XIi1Ss.png"><br>发现没有任意来源了，<br>于是搜索了下，可以直接在当前环境下开启，这个是区别于之前的经验的，因为之前关闭 <code>spctl</code> 好像是需要进恢复模式的<br>我试了下在 mac os 14.3 中目前是可以直接把这个关闭掉</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sudo spctl --master-disable</span><br></pre></td></tr></table></figure>
|
|
<p>关闭后重新打开系统设置<br>就能看到任意来源了<br><img data-src="https://img.nicksxs.me/blog/YI2P6L.png"><br>捎带解释下这个 <code>spctl</code> 的意思是 <code>System Policy Control</code><br>用于管理系统策略控制。 禁用Gatekeeper的主开关，允许运行任何来源的应用。<br>此命令会降低Mac的安全性，因为它允许运行未经验证的应用。 启用后，需小心下载和运行应用。<br>所以还是需要小心关闭，一般在确认来源可靠才能关闭</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>mac</category>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>mac</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>minimum-size-subarray-sum-209</title>
|
|
<url>/2016/10/11/minimum-size-subarray-sum-209/</url>
|
|
<content><![CDATA[<h3 id="problem"><a href="#problem" class="headerlink" title="problem"></a>problem</h3><p>Given an array of n positive integers and a positive integer s, find the minimal length of a subarray of which the sum ≥ s. If there isn’t one, return 0 instead. </p>
|
|
<p>For example, given the array <code>[2,3,1,2,4,3]</code> and <code>s = 7</code>,<br>the subarray <code>[4,3]</code> has the minimal length under the problem constraint. </p>
|
|
<h4 id="题解"><a href="#题解" class="headerlink" title="题解"></a>题解</h4><p>参考，滑动窗口，跟之前Data Structure课上的online算法有点像，<a href="http://blog.csdn.net/lisonglisonglisong/article/details/45666975">链接</a> </p>
|
|
<h4 id="Code"><a href="#Code" class="headerlink" title="Code"></a>Code</h4><figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">Solution</span> {</span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line"> <span class="function"><span class="type">int</span> <span class="title">minSubArrayLen</span><span class="params">(<span class="type">int</span> s, vector<<span class="type">int</span>>& nums)</span> </span>{</span><br><span class="line"> <span class="type">int</span> len = nums.<span class="built_in">size</span>();</span><br><span class="line"> <span class="keyword">if</span>(len == <span class="number">0</span>) <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line"> <span class="type">int</span> minlen = INT_MAX;</span><br><span class="line"> <span class="type">int</span> sum = <span class="number">0</span>;</span><br><span class="line"> </span><br><span class="line"> <span class="type">int</span> left = <span class="number">0</span>;</span><br><span class="line"> <span class="type">int</span> right = <span class="number">-1</span>;</span><br><span class="line"> <span class="keyword">while</span>(right < len)</span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">while</span>(sum < s && right < len)</span><br><span class="line"> sum += nums[++right];</span><br><span class="line"> <span class="keyword">if</span>(sum >= s)</span><br><span class="line"> {</span><br><span class="line"> minlen = minlen < right - left + <span class="number">1</span> ? minlen : right - left + <span class="number">1</span>;</span><br><span class="line"> sum -= nums[left++];</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> minlen > len ? <span class="number">0</span> : minlen;</span><br><span class="line"> }</span><br><span class="line">};</span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>c++</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>C++ 指针使用中的一个小问题</title>
|
|
<url>/2014/12/23/my-new-post/</url>
|
|
<content><![CDATA[<h4 id="在工作中碰到的一点C-指针上的一点小问题"><a href="#在工作中碰到的一点C-指针上的一点小问题" class="headerlink" title="在工作中碰到的一点C++指针上的一点小问题"></a>在工作中碰到的一点C++指针上的一点小问题</h4><hr>
|
|
<p>在C++中，应该是从C语言就开始了，除了<a href="">void</a>型指针之外都是需要有分配对应的内存才可以使用，同时<a href="">malloc</a>与<a href="">free</a>成对使用，<a href="">new</a>与<a href="">delete</a>成对使用，否则造成内存泄漏。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>C++</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>博客，文章</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>mybatis 的 foreach 使用的注意点</title>
|
|
<url>/2022/07/09/mybatis-%E7%9A%84-foreach-%E4%BD%BF%E7%94%A8%E7%9A%84%E6%B3%A8%E6%84%8F%E7%82%B9/</url>
|
|
<content><![CDATA[<p>mybatis 在作为轻量级 orm 框架，如果要使用类似于 in 查询的语句，除了直接替换字符串，还可以使用 foreach 标签<br>在mybatis的 dtd 文件中可以看到可以配置这些字段，</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line"><!ELEMENT foreach (#PCDATA | include | trim | where | set | foreach | choose | if | bind)*></span><br><span class="line"><!ATTLIST foreach</span><br><span class="line">collection CDATA #REQUIRED</span><br><span class="line">item CDATA #IMPLIED</span><br><span class="line">index CDATA #IMPLIED</span><br><span class="line">open CDATA #IMPLIED</span><br><span class="line">close CDATA #IMPLIED</span><br><span class="line">separator CDATA #IMPLIED</span><br><span class="line">></span><br></pre></td></tr></table></figure>
|
|
<p>collection 表示需要使用 foreach 的集合，item 表示进行迭代的变量名，index 就是索引值，而 open 跟 close<br>代表拼接的起始和结束符号，一般就是左右括号，separator 则是每个 item 直接的分隔符</p>
|
|
<p>例如写了一个简单的 sql 查询</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">select</span> <span class="attr">id</span>=<span class="string">"search"</span> <span class="attr">parameterType</span>=<span class="string">"list"</span> <span class="attr">resultMap</span>=<span class="string">"StudentMap"</span>></span></span><br><span class="line"> select * from student</span><br><span class="line"> <span class="tag"><<span class="name">where</span>></span></span><br><span class="line"> id in</span><br><span class="line"> <span class="tag"><<span class="name">foreach</span> <span class="attr">collection</span>=<span class="string">"list"</span> <span class="attr">item</span>=<span class="string">"item"</span> <span class="attr">open</span>=<span class="string">"("</span> <span class="attr">close</span>=<span class="string">")"</span> <span class="attr">separator</span>=<span class="string">","</span>></span></span><br><span class="line"> #{item}</span><br><span class="line"> <span class="tag"></<span class="name">foreach</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">where</span>></span></span><br><span class="line"><span class="tag"></<span class="name">select</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>这里就发现了一个问题，collection 对应的这个值，如果传入的参数是个 HashMap，collection 的这个值就是以此作为<br> key 从这个 HashMap 获取对应的集合，但是这里有几个特殊的小技巧，<br> 在上面的这个方法对应的接口方法定义中</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> List<Student> <span class="title function_">search</span><span class="params">(List<Long> userIds)</span>;</span><br></pre></td></tr></table></figure>
|
|
<p>我是这么定义的，而 collection 的值是<code>list</code>，这里就有一点不能理解了，但其实是 mybatis 考虑到使用的方便性，<br>帮我们做了一点点小转换，我们翻一下 mybatis 的DefaultSqlSession 中的代码可以看到</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <E> List<E> <span class="title function_">selectList</span><span class="params">(String statement, Object parameter, RowBounds rowBounds)</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">MappedStatement</span> <span class="variable">ms</span> <span class="operator">=</span> configuration.getMappedStatement(statement);</span><br><span class="line"> <span class="keyword">return</span> executor.query(ms, wrapCollection(parameter), rowBounds, Executor.NO_RESULT_HANDLER);</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> ExceptionFactory.wrapException(<span class="string">"Error querying database. Cause: "</span> + e, e);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> ErrorContext.instance().reset();</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"><span class="comment">// 就是在这帮我们做了转换</span></span><br><span class="line"> <span class="keyword">private</span> Object <span class="title function_">wrapCollection</span><span class="params">(<span class="keyword">final</span> Object object)</span> {</span><br><span class="line"> <span class="keyword">if</span> (object <span class="keyword">instanceof</span> Collection) {</span><br><span class="line"> StrictMap<Object> map = <span class="keyword">new</span> <span class="title class_">StrictMap</span><Object>();</span><br><span class="line"> map.put(<span class="string">"collection"</span>, object);</span><br><span class="line"> <span class="keyword">if</span> (object <span class="keyword">instanceof</span> List) {</span><br><span class="line"> <span class="comment">// 如果类型是list 就会转成以 list 为 key 的 map</span></span><br><span class="line"> map.put(<span class="string">"list"</span>, object);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> map;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (object != <span class="literal">null</span> && object.getClass().isArray()) {</span><br><span class="line"> StrictMap<Object> map = <span class="keyword">new</span> <span class="title class_">StrictMap</span><Object>();</span><br><span class="line"> map.put(<span class="string">"array"</span>, object);</span><br><span class="line"> <span class="keyword">return</span> map;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> object;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Mybatis</category>
|
|
<category>Mysql</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Mysql</tag>
|
|
<tag>Mybatis</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>mybatis 的 $ 和 # 是有啥区别</title>
|
|
<url>/2020/09/06/mybatis-%E7%9A%84-%E5%92%8C-%E6%98%AF%E6%9C%89%E5%95%A5%E5%8C%BA%E5%88%AB/</url>
|
|
<content><![CDATA[<p>这个问题也是面试中常被问到的，就抽空来了解下这个，跳过一大段前面初始化的逻辑，<br>对于一条<code>select * from t1 where id = #{id}</code>这样的 sql，在初始化扫描 mapper 的xml文件的时候会根据是否是 dynamic 来判断生成 DynamicSqlSource 还是 RawSqlSource，这里它是一条 RawSqlSource，<br>在这里做了替换，将<code>#{}</code>替换成了<code>?</code><br><img data-src="https://img.nicksxs.me/uPic/c84r0g.png"><br>前面说的是否 dynamic 就是在这里进行判断<br><img data-src="https://img.nicksxs.me/uPic/E0ODcp.png"></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// org.apache.ibatis.scripting.xmltags.XMLScriptBuilder#parseScriptNode</span></span><br><span class="line"><span class="keyword">public</span> SqlSource <span class="title function_">parseScriptNode</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">MixedSqlNode</span> <span class="variable">rootSqlNode</span> <span class="operator">=</span> parseDynamicTags(context);</span><br><span class="line"> SqlSource sqlSource;</span><br><span class="line"> <span class="keyword">if</span> (isDynamic) {</span><br><span class="line"> sqlSource = <span class="keyword">new</span> <span class="title class_">DynamicSqlSource</span>(configuration, rootSqlNode);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> sqlSource = <span class="keyword">new</span> <span class="title class_">RawSqlSource</span>(configuration, rootSqlNode, parameterType);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> sqlSource;</span><br><span class="line"> }</span><br><span class="line"><span class="comment">// org.apache.ibatis.scripting.xmltags.XMLScriptBuilder#parseDynamicTags</span></span><br><span class="line"><span class="keyword">protected</span> MixedSqlNode <span class="title function_">parseDynamicTags</span><span class="params">(XNode node)</span> {</span><br><span class="line"> List<SqlNode> contents = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"> <span class="type">NodeList</span> <span class="variable">children</span> <span class="operator">=</span> node.getNode().getChildNodes();</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < children.getLength(); i++) {</span><br><span class="line"> <span class="type">XNode</span> <span class="variable">child</span> <span class="operator">=</span> node.newXNode(children.item(i));</span><br><span class="line"> <span class="keyword">if</span> (child.getNode().getNodeType() == Node.CDATA_SECTION_NODE || child.getNode().getNodeType() == Node.TEXT_NODE) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">data</span> <span class="operator">=</span> child.getStringBody(<span class="string">""</span>);</span><br><span class="line"> <span class="type">TextSqlNode</span> <span class="variable">textSqlNode</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">TextSqlNode</span>(data);</span><br><span class="line"> <span class="keyword">if</span> (textSqlNode.isDynamic()) {</span><br><span class="line"> contents.add(textSqlNode);</span><br><span class="line"> isDynamic = <span class="literal">true</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> contents.add(<span class="keyword">new</span> <span class="title class_">StaticTextSqlNode</span>(data));</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (child.getNode().getNodeType() == Node.ELEMENT_NODE) { <span class="comment">// issue #628</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">nodeName</span> <span class="operator">=</span> child.getNode().getNodeName();</span><br><span class="line"> <span class="type">NodeHandler</span> <span class="variable">handler</span> <span class="operator">=</span> nodeHandlerMap.get(nodeName);</span><br><span class="line"> <span class="keyword">if</span> (handler == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">BuilderException</span>(<span class="string">"Unknown element <"</span> + nodeName + <span class="string">"> in SQL statement."</span>);</span><br><span class="line"> }</span><br><span class="line"> handler.handleNode(child, contents);</span><br><span class="line"> isDynamic = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">MixedSqlNode</span>(contents);</span><br><span class="line"> }</span><br><span class="line"><span class="comment">// org.apache.ibatis.scripting.xmltags.TextSqlNode#isDynamic</span></span><br><span class="line"> <span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">isDynamic</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">DynamicCheckerTokenParser</span> <span class="variable">checker</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">DynamicCheckerTokenParser</span>();</span><br><span class="line"> <span class="type">GenericTokenParser</span> <span class="variable">parser</span> <span class="operator">=</span> createParser(checker);</span><br><span class="line"> parser.parse(text);</span><br><span class="line"> <span class="keyword">return</span> checker.isDynamic();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">private</span> GenericTokenParser <span class="title function_">createParser</span><span class="params">(TokenHandler handler)</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">GenericTokenParser</span>(<span class="string">"${"</span>, <span class="string">"}"</span>, handler);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>可以看到其中一个条件就是是否有<code>${}</code>这种占位符，假如说上面的 sql 换成 <code>${}</code>，那么可以看到它会在这里创建一个 dynamicSqlSource,</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// org.apache.ibatis.scripting.xmltags.DynamicSqlSource</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">DynamicSqlSource</span> <span class="keyword">implements</span> <span class="title class_">SqlSource</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> Configuration configuration;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> SqlNode rootSqlNode;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">DynamicSqlSource</span><span class="params">(Configuration configuration, SqlNode rootSqlNode)</span> {</span><br><span class="line"> <span class="built_in">this</span>.configuration = configuration;</span><br><span class="line"> <span class="built_in">this</span>.rootSqlNode = rootSqlNode;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> BoundSql <span class="title function_">getBoundSql</span><span class="params">(Object parameterObject)</span> {</span><br><span class="line"> <span class="type">DynamicContext</span> <span class="variable">context</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">DynamicContext</span>(configuration, parameterObject);</span><br><span class="line"> rootSqlNode.apply(context);</span><br><span class="line"> <span class="type">SqlSourceBuilder</span> <span class="variable">sqlSourceParser</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SqlSourceBuilder</span>(configuration);</span><br><span class="line"> Class<?> parameterType = parameterObject == <span class="literal">null</span> ? Object.class : parameterObject.getClass();</span><br><span class="line"> <span class="type">SqlSource</span> <span class="variable">sqlSource</span> <span class="operator">=</span> sqlSourceParser.parse(context.getSql(), parameterType, context.getBindings());</span><br><span class="line"> <span class="type">BoundSql</span> <span class="variable">boundSql</span> <span class="operator">=</span> sqlSource.getBoundSql(parameterObject);</span><br><span class="line"> context.getBindings().forEach(boundSql::setAdditionalParameter);</span><br><span class="line"> <span class="keyword">return</span> boundSql;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
<p>这里眼尖的同学可能就可以看出来了，RawSqlSource 在初始化的时候已经经过了 parse，把<code>#{}</code>替换成了<code>?</code>占位符，但是 DynamicSqlSource 并没有<br><img data-src="https://img.nicksxs.me/uPic/dnZWP7.png">再看这个图，我们发现在这的时候还没有进行替换<br>然后往里跟<br><img data-src="https://img.nicksxs.me/uPic/YGVD0P.png">好像是这里了<br><img data-src="https://img.nicksxs.me/uPic/BWHuUO.png"><br>这里 rootSqlNode.apply 其实是一个对原来 sql 的解析结果的一个循环调用，不同类型的标签会构成不同的 node，像这里就是一个 textSqlNode<br><img data-src="https://img.nicksxs.me/uPic/eTiQm6.png"><br>可以发现到这我们的 sql 已经被替换了，而且是直接作为 string 类型替换的，所以可以明白了这个问题所在，就是注入，不过细心的同学发现其实这里是有个<br><img data-src="https://img.nicksxs.me/uPic/Cc4JPw.png"><br>理论上还是可以做过滤的，不过好像现在没用起来。<br>我们前面可以发现对于<code>#{}</code>是在启动扫描 mapper的 xml 文件就替换成了 <code>?</code>,然后是在什么时候变成实际的值的呢<br><img data-src="https://img.nicksxs.me/uPic/fRHpNO.png"><br>发现到这的时候还是没有替换，其实说白了也就是 prepareStatement 那一套，<br><img data-src="https://img.nicksxs.me/uPic/4tNU7f.png"><br>在这里进行替换，会拿到 org.apache.ibatis.mapping.ParameterMapping，然后进行替换，因为会带着类型信息，所以不用担心注入咯</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Mybatis</category>
|
|
<category>Mysql</category>
|
|
<category>Spring</category>
|
|
<category>Sql注入</category>
|
|
<category>Mybatis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Mysql</tag>
|
|
<tag>Mybatis</tag>
|
|
<tag>Sql注入</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>mybatis 的缓存是怎么回事</title>
|
|
<url>/2020/10/03/mybatis-%E7%9A%84%E7%BC%93%E5%AD%98%E6%98%AF%E6%80%8E%E4%B9%88%E5%9B%9E%E4%BA%8B/</url>
|
|
<content><![CDATA[<p>Java 真的是任何一个中间件，比较常用的那种，都有很多内容值得深挖，比如这个缓存，慢慢有过一些感悟，比如如何提升性能，缓存无疑是一大重要手段，最底层开始 CPU 就有缓存，而且又小又贵，再往上一点内存一般作为硬盘存储在运行时的存储，一般在代码里也会用内存作为一些本地缓存，譬如数据库，像 mysql 这种也是有innodb_buffer_pool来提升查询效率，本质上理解就是用更快的存储作为相对慢存储的缓存，减少查询直接访问较慢的存储，并且这个都是相对的，比起 cpu 的缓存，那内存也是渣，但是与普通机械硬盘相比，那也是两个次元的水平。</p>
|
|
<p>闲扯这么多来说说 mybatis 的缓存，mybatis 一般作为一个轻量级的 orm 使用，相对应的就是比较重量级的 hibernate，不过不在这次讨论范围，上一次是主要讲了 mybatis 在解析 sql 过程中，对于两种占位符的不同替换实现策略，这次主要聊下 mybatis 的缓存，前面其实得了解下前置的东西，比如 sqlsession，先当做我们知道 sqlsession 是个什么玩意，可能或多或少的知道 mybatis 是有两级缓存，</p>
|
|
<h3 id="一级缓存"><a href="#一级缓存" class="headerlink" title="一级缓存"></a>一级缓存</h3><p>第一级的缓存是在 <a href="https://github.com/mybatis/mybatis-3/blob/master/src/main/java/org/apache/ibatis/executor/BaseExecutor.java#L65">BaseExecutor</a> 中的 <a href="https://github.com/mybatis/mybatis-3/blob/master/src/main/java/org/apache/ibatis/cache/impl/PerpetualCache.java#L27">PerpetualCache</a>，它是个最基本的缓存实现类，使用了 HashMap 实现缓存功能，代码其实没几十行</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">PerpetualCache</span> <span class="keyword">implements</span> <span class="title class_">Cache</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> String id;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> Map<Object, Object> cache = <span class="keyword">new</span> <span class="title class_">HashMap</span><>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">PerpetualCache</span><span class="params">(String id)</span> {</span><br><span class="line"> <span class="built_in">this</span>.id = id;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> String <span class="title function_">getId</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> id;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="type">int</span> <span class="title function_">getSize</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> cache.size();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">putObject</span><span class="params">(Object key, Object value)</span> {</span><br><span class="line"> cache.put(key, value);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> Object <span class="title function_">getObject</span><span class="params">(Object key)</span> {</span><br><span class="line"> <span class="keyword">return</span> cache.get(key);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> Object <span class="title function_">removeObject</span><span class="params">(Object key)</span> {</span><br><span class="line"> <span class="keyword">return</span> cache.remove(key);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">clear</span><span class="params">()</span> {</span><br><span class="line"> cache.clear();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">equals</span><span class="params">(Object o)</span> {</span><br><span class="line"> <span class="keyword">if</span> (getId() == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">CacheException</span>(<span class="string">"Cache instances require an ID."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span> == o) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (!(o <span class="keyword">instanceof</span> Cache)) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">Cache</span> <span class="variable">otherCache</span> <span class="operator">=</span> (Cache) o;</span><br><span class="line"> <span class="keyword">return</span> getId().equals(otherCache.getId());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="type">int</span> <span class="title function_">hashCode</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (getId() == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">CacheException</span>(<span class="string">"Cache instances require an ID."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> getId().hashCode();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>可以看一下BaseExecutor 的构造函数</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="title function_">BaseExecutor</span><span class="params">(Configuration configuration, Transaction transaction)</span> {</span><br><span class="line"> <span class="built_in">this</span>.transaction = transaction;</span><br><span class="line"> <span class="built_in">this</span>.deferredLoads = <span class="keyword">new</span> <span class="title class_">ConcurrentLinkedQueue</span><>();</span><br><span class="line"> <span class="built_in">this</span>.localCache = <span class="keyword">new</span> <span class="title class_">PerpetualCache</span>(<span class="string">"LocalCache"</span>);</span><br><span class="line"> <span class="built_in">this</span>.localOutputParameterCache = <span class="keyword">new</span> <span class="title class_">PerpetualCache</span>(<span class="string">"LocalOutputParameterCache"</span>);</span><br><span class="line"> <span class="built_in">this</span>.closed = <span class="literal">false</span>;</span><br><span class="line"> <span class="built_in">this</span>.configuration = configuration;</span><br><span class="line"> <span class="built_in">this</span>.wrapper = <span class="built_in">this</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>就是把 PerpetualCache 作为 localCache，然后怎么使用我看简单看一下，BaseExecutor 的查询首先是调用这个函数</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <E> List<E> <span class="title function_">query</span><span class="params">(MappedStatement ms, Object parameter, RowBounds rowBounds, ResultHandler resultHandler)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="type">BoundSql</span> <span class="variable">boundSql</span> <span class="operator">=</span> ms.getBoundSql(parameter);</span><br><span class="line"> <span class="type">CacheKey</span> <span class="variable">key</span> <span class="operator">=</span> createCacheKey(ms, parameter, rowBounds, boundSql);</span><br><span class="line"> <span class="keyword">return</span> query(ms, parameter, rowBounds, resultHandler, key, boundSql);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>可以看到首先是调用了 <a href="https://github.com/mybatis/mybatis-3/blob/master/src/main/java/org/apache/ibatis/executor/BaseExecutor.java#L195">createCacheKey</a> 方法，这个方法呢，先不看怎么写的，如果我们自己要实现这么个缓存，首先这个缓存 key 的设计也是个问题，如果是以表名加主键作为 key，那么分页查询，或者没有主键的时候就不行，来看看 mybatis 是怎么设计的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> CacheKey <span class="title function_">createCacheKey</span><span class="params">(MappedStatement ms, Object parameterObject, RowBounds rowBounds, BoundSql boundSql)</span> {</span><br><span class="line"> <span class="keyword">if</span> (closed) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">ExecutorException</span>(<span class="string">"Executor was closed."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="type">CacheKey</span> <span class="variable">cacheKey</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">CacheKey</span>();</span><br><span class="line"> cacheKey.update(ms.getId());</span><br><span class="line"> cacheKey.update(rowBounds.getOffset());</span><br><span class="line"> cacheKey.update(rowBounds.getLimit());</span><br><span class="line"> cacheKey.update(boundSql.getSql());</span><br><span class="line"> List<ParameterMapping> parameterMappings = boundSql.getParameterMappings();</span><br><span class="line"> <span class="type">TypeHandlerRegistry</span> <span class="variable">typeHandlerRegistry</span> <span class="operator">=</span> ms.getConfiguration().getTypeHandlerRegistry();</span><br><span class="line"> <span class="comment">// mimic DefaultParameterHandler logic</span></span><br><span class="line"> <span class="keyword">for</span> (ParameterMapping parameterMapping : parameterMappings) {</span><br><span class="line"> <span class="keyword">if</span> (parameterMapping.getMode() != ParameterMode.OUT) {</span><br><span class="line"> Object value;</span><br><span class="line"> <span class="type">String</span> <span class="variable">propertyName</span> <span class="operator">=</span> parameterMapping.getProperty();</span><br><span class="line"> <span class="keyword">if</span> (boundSql.hasAdditionalParameter(propertyName)) {</span><br><span class="line"> value = boundSql.getAdditionalParameter(propertyName);</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (parameterObject == <span class="literal">null</span>) {</span><br><span class="line"> value = <span class="literal">null</span>;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (typeHandlerRegistry.hasTypeHandler(parameterObject.getClass())) {</span><br><span class="line"> value = parameterObject;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="type">MetaObject</span> <span class="variable">metaObject</span> <span class="operator">=</span> configuration.newMetaObject(parameterObject);</span><br><span class="line"> value = metaObject.getValue(propertyName);</span><br><span class="line"> }</span><br><span class="line"> cacheKey.update(value);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (configuration.getEnvironment() != <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// issue #176</span></span><br><span class="line"> cacheKey.update(configuration.getEnvironment().getId());</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> cacheKey;</span><br><span class="line"> }</span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
<p>首先需要 id，这个 id 是 mapper 里方法的 id， 然后是偏移量跟返回行数，再就是 sql，然后是参数，基本上是会有影响的都加进去了，在这个 update 里面</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">update</span><span class="params">(Object object)</span> {</span><br><span class="line"> <span class="type">int</span> <span class="variable">baseHashCode</span> <span class="operator">=</span> object == <span class="literal">null</span> ? <span class="number">1</span> : ArrayUtil.hashCode(object);</span><br><span class="line"></span><br><span class="line"> count++;</span><br><span class="line"> checksum += baseHashCode;</span><br><span class="line"> baseHashCode *= count;</span><br><span class="line"></span><br><span class="line"> hashcode = multiplier * hashcode + baseHashCode;</span><br><span class="line"></span><br><span class="line"> updateList.add(object);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>其实是一个 hash 转换，具体不纠结，就是提高特异性，然后回来就是继续调用 query</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <E> List<E> <span class="title function_">query</span><span class="params">(MappedStatement ms, Object parameter, RowBounds rowBounds, ResultHandler resultHandler, CacheKey key, BoundSql boundSql)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> ErrorContext.instance().resource(ms.getResource()).activity(<span class="string">"executing a query"</span>).object(ms.getId());</span><br><span class="line"> <span class="keyword">if</span> (closed) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">ExecutorException</span>(<span class="string">"Executor was closed."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (queryStack == <span class="number">0</span> && ms.isFlushCacheRequired()) {</span><br><span class="line"> clearLocalCache();</span><br><span class="line"> }</span><br><span class="line"> List<E> list;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> queryStack++;</span><br><span class="line"> list = resultHandler == <span class="literal">null</span> ? (List<E>) localCache.getObject(key) : <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">if</span> (list != <span class="literal">null</span>) {</span><br><span class="line"> handleLocallyCachedOutputParameters(ms, key, parameter, boundSql);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> list = queryFromDatabase(ms, parameter, rowBounds, resultHandler, key, boundSql);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> queryStack--;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (queryStack == <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">for</span> (DeferredLoad deferredLoad : deferredLoads) {</span><br><span class="line"> deferredLoad.load();</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// issue #601</span></span><br><span class="line"> deferredLoads.clear();</span><br><span class="line"> <span class="keyword">if</span> (configuration.getLocalCacheScope() == LocalCacheScope.STATEMENT) {</span><br><span class="line"> <span class="comment">// issue #482</span></span><br><span class="line"> clearLocalCache();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> list;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>可以看到是先从 localCache 里取，取不到再 queryFromDatabase，其实比较简单，这是一级缓存，考虑到 sqlsession 跟 BaseExecutor 的关系，其实是随着 sqlsession 来保证这个缓存不会出现脏数据幻读的情况，当然事务相关的后面可能再单独聊。</p>
|
|
<h3 id="二级缓存"><a href="#二级缓存" class="headerlink" title="二级缓存"></a>二级缓存</h3><p>其实这个一级二级顺序有点反过来，其实查询的是先走的二级缓存，当然二级的需要配置开启，默认不开，<br>需要通过</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">setting</span> <span class="attr">name</span>=<span class="string">"cacheEnabled"</span> <span class="attr">value</span>=<span class="string">"true"</span>/></span></span><br></pre></td></tr></table></figure>
|
|
<p>来开启，然后我们的查询就会走到</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">CachingExecutor</span> <span class="keyword">implements</span> <span class="title class_">Executor</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> Executor delegate;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="type">TransactionalCacheManager</span> <span class="variable">tcm</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">TransactionalCacheManager</span>();</span><br></pre></td></tr></table></figure>
|
|
<p>这个 Executor 中，这里我放了类里面的元素，发现没有一个 Cache 类，这就是一个特点了，往下看查询过程</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <E> List<E> <span class="title function_">query</span><span class="params">(MappedStatement ms, Object parameterObject, RowBounds rowBounds, ResultHandler resultHandler)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="type">BoundSql</span> <span class="variable">boundSql</span> <span class="operator">=</span> ms.getBoundSql(parameterObject);</span><br><span class="line"> <span class="type">CacheKey</span> <span class="variable">key</span> <span class="operator">=</span> createCacheKey(ms, parameterObject, rowBounds, boundSql);</span><br><span class="line"> <span class="keyword">return</span> query(ms, parameterObject, rowBounds, resultHandler, key, boundSql);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <E> List<E> <span class="title function_">query</span><span class="params">(MappedStatement ms, Object parameterObject, RowBounds rowBounds, ResultHandler resultHandler, CacheKey key, BoundSql boundSql)</span></span><br><span class="line"> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="type">Cache</span> <span class="variable">cache</span> <span class="operator">=</span> ms.getCache();</span><br><span class="line"> <span class="keyword">if</span> (cache != <span class="literal">null</span>) {</span><br><span class="line"> flushCacheIfRequired(ms);</span><br><span class="line"> <span class="keyword">if</span> (ms.isUseCache() && resultHandler == <span class="literal">null</span>) {</span><br><span class="line"> ensureNoOutParams(ms, boundSql);</span><br><span class="line"> <span class="meta">@SuppressWarnings("unchecked")</span></span><br><span class="line"> List<E> list = (List<E>) tcm.getObject(cache, key);</span><br><span class="line"> <span class="keyword">if</span> (list == <span class="literal">null</span>) {</span><br><span class="line"> list = delegate.query(ms, parameterObject, rowBounds, resultHandler, key, boundSql);</span><br><span class="line"> tcm.putObject(cache, key, list); <span class="comment">// issue #578 and #116</span></span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> list;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> delegate.query(ms, parameterObject, rowBounds, resultHandler, key, boundSql);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>看到没，其实缓存是从 tcm 这个成员变量里取，而这个是什么呢，事务性缓存(直译下)，因为这个其实是用 MappedStatement 里的 Cache 作为key 从 tcm 的 map 取出来的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">TransactionalCacheManager</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> Map<Cache, TransactionalCache> transactionalCaches = <span class="keyword">new</span> <span class="title class_">HashMap</span><>();</span><br></pre></td></tr></table></figure>
|
|
<p>MappedStatement是被全局使用的，所以其实二级缓存是跟着 mapper 的 namespace 走的，可以被多个 CachingExecutor 获取到，就会出现线程安全问题，线程安全问题可以用SynchronizedCache来解决，就是加锁，但是对于事务中的脏读，使用了TransactionalCache来解决这个问题，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">TransactionalCache</span> <span class="keyword">implements</span> <span class="title class_">Cache</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">Log</span> <span class="variable">log</span> <span class="operator">=</span> LogFactory.getLog(TransactionalCache.class);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> Cache delegate;</span><br><span class="line"> <span class="keyword">private</span> <span class="type">boolean</span> clearOnCommit;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> Map<Object, Object> entriesToAddOnCommit;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> Set<Object> entriesMissedInCache;</span><br></pre></td></tr></table></figure>
|
|
<p>在事务还没提交的时候，会把中间状态的数据放在 entriesToAddOnCommit 中，只有在提交后会放进共享缓存中，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">commit</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (clearOnCommit) {</span><br><span class="line"> delegate.clear();</span><br><span class="line"> }</span><br><span class="line"> flushPendingEntries();</span><br><span class="line"> reset();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Mybatis</category>
|
|
<category>Mybatis</category>
|
|
<category>Spring</category>
|
|
<category>缓存</category>
|
|
<category>Mybatis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Mysql</tag>
|
|
<tag>Mybatis</tag>
|
|
<tag>缓存</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>mybatis系列-mybatis是如何初始化mapper的</title>
|
|
<url>/2022/12/04/mybatis%E6%98%AF%E5%A6%82%E4%BD%95%E5%88%9D%E5%A7%8B%E5%8C%96mapper%E7%9A%84/</url>
|
|
<content><![CDATA[<p>前一篇讲了mybatis的初始化使用，如果我第一次看到这个使用入门文档，比较会产生疑惑的是配置了mapper，怎么就能通过<code>selectOne</code>跟语句id就能执行sql了，那么第一个问题，就是mapper是怎么被解析的，存在哪里，怎么被拿出来的</p>
|
|
<h2 id="添加解析mapper"><a href="#添加解析mapper" class="headerlink" title="添加解析mapper"></a>添加解析mapper</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line">org.apache.ibatis.session.SqlSessionFactoryBuilder#build(java.io.InputStream)</span><br><span class="line"><span class="keyword">public</span> SqlSessionFactory <span class="title function_">build</span><span class="params">(InputStream inputStream)</span> {</span><br><span class="line"> <span class="keyword">return</span> build(inputStream, <span class="literal">null</span>, <span class="literal">null</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>通过读取mybatis-config.xml来构建SqlSessionFactory，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> SqlSessionFactory <span class="title function_">build</span><span class="params">(InputStream inputStream, String environment, Properties properties)</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 创建下xml的解析器</span></span><br><span class="line"> <span class="type">XMLConfigBuilder</span> <span class="variable">parser</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">XMLConfigBuilder</span>(inputStream, environment, properties);</span><br><span class="line"> <span class="comment">// 进行解析，后再构建</span></span><br><span class="line"> <span class="keyword">return</span> build(parser.parse());</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> ExceptionFactory.wrapException(<span class="string">"Error building SqlSession."</span>, e);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> ErrorContext.instance().reset();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (inputStream != <span class="literal">null</span>) {</span><br><span class="line"> inputStream.close();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (IOException e) {</span><br><span class="line"> <span class="comment">// Intentionally ignore. Prefer previous error.</span></span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>创建XMLConfigBuilder</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">XMLConfigBuilder</span><span class="params">(InputStream inputStream, String environment, Properties props)</span> {</span><br><span class="line"> <span class="comment">// --------> 创建 XPathParser</span></span><br><span class="line"> <span class="built_in">this</span>(<span class="keyword">new</span> <span class="title class_">XPathParser</span>(inputStream, <span class="literal">true</span>, props, <span class="keyword">new</span> <span class="title class_">XMLMapperEntityResolver</span>()), environment, props);</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="title function_">XPathParser</span><span class="params">(InputStream inputStream, <span class="type">boolean</span> validation, Properties variables, EntityResolver entityResolver)</span> {</span><br><span class="line"> commonConstructor(validation, variables, entityResolver);</span><br><span class="line"> <span class="built_in">this</span>.document = createDocument(<span class="keyword">new</span> <span class="title class_">InputSource</span>(inputStream));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"><span class="keyword">private</span> <span class="title function_">XMLConfigBuilder</span><span class="params">(XPathParser parser, String environment, Properties props)</span> {</span><br><span class="line"> <span class="built_in">super</span>(<span class="keyword">new</span> <span class="title class_">Configuration</span>());</span><br><span class="line"> ErrorContext.instance().resource(<span class="string">"SQL Mapper Configuration"</span>);</span><br><span class="line"> <span class="built_in">this</span>.configuration.setVariables(props);</span><br><span class="line"> <span class="built_in">this</span>.parsed = <span class="literal">false</span>;</span><br><span class="line"> <span class="built_in">this</span>.environment = environment;</span><br><span class="line"> <span class="built_in">this</span>.parser = parser;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这里主要是创建了Builder包含了Parser<br>然后调用parse方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Configuration <span class="title function_">parse</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (parsed) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">BuilderException</span>(<span class="string">"Each XMLConfigBuilder can only be used once."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 标记下是否已解析，但是这里是否有线程安全问题</span></span><br><span class="line"> parsed = <span class="literal">true</span>;</span><br><span class="line"> <span class="comment">// --------> 解析配置</span></span><br><span class="line"> parseConfiguration(parser.evalNode(<span class="string">"/configuration"</span>));</span><br><span class="line"> <span class="keyword">return</span> configuration;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>实际的解析区分了各类标签</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">parseConfiguration</span><span class="params">(XNode root)</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// issue #117 read properties first</span></span><br><span class="line"> <span class="comment">// 解析properties，这个不是spring自带的，需要额外配置，并且在config文件里应该放在最前</span></span><br><span class="line"> propertiesElement(root.evalNode(<span class="string">"properties"</span>));</span><br><span class="line"> <span class="type">Properties</span> <span class="variable">settings</span> <span class="operator">=</span> settingsAsProperties(root.evalNode(<span class="string">"settings"</span>));</span><br><span class="line"> loadCustomVfs(settings);</span><br><span class="line"> loadCustomLogImpl(settings);</span><br><span class="line"> typeAliasesElement(root.evalNode(<span class="string">"typeAliases"</span>));</span><br><span class="line"> pluginElement(root.evalNode(<span class="string">"plugins"</span>));</span><br><span class="line"> objectFactoryElement(root.evalNode(<span class="string">"objectFactory"</span>));</span><br><span class="line"> objectWrapperFactoryElement(root.evalNode(<span class="string">"objectWrapperFactory"</span>));</span><br><span class="line"> reflectorFactoryElement(root.evalNode(<span class="string">"reflectorFactory"</span>));</span><br><span class="line"> settingsElement(settings);</span><br><span class="line"> <span class="comment">// read it after objectFactory and objectWrapperFactory issue #631</span></span><br><span class="line"> environmentsElement(root.evalNode(<span class="string">"environments"</span>));</span><br><span class="line"> databaseIdProviderElement(root.evalNode(<span class="string">"databaseIdProvider"</span>));</span><br><span class="line"> typeHandlerElement(root.evalNode(<span class="string">"typeHandlers"</span>));</span><br><span class="line"> <span class="comment">// ----------> 我们需要关注的是mapper的处理</span></span><br><span class="line"> mapperElement(root.evalNode(<span class="string">"mappers"</span>));</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">BuilderException</span>(<span class="string">"Error parsing SQL Mapper Configuration. Cause: "</span> + e, e);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后就是调用到mapperElement方法了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">mapperElement</span><span class="params">(XNode parent)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="keyword">if</span> (parent != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">for</span> (XNode child : parent.getChildren()) {</span><br><span class="line"> <span class="keyword">if</span> (<span class="string">"package"</span>.equals(child.getName())) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">mapperPackage</span> <span class="operator">=</span> child.getStringAttribute(<span class="string">"name"</span>);</span><br><span class="line"> configuration.addMappers(mapperPackage);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">resource</span> <span class="operator">=</span> child.getStringAttribute(<span class="string">"resource"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">url</span> <span class="operator">=</span> child.getStringAttribute(<span class="string">"url"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">mapperClass</span> <span class="operator">=</span> child.getStringAttribute(<span class="string">"class"</span>);</span><br><span class="line"> <span class="keyword">if</span> (resource != <span class="literal">null</span> && url == <span class="literal">null</span> && mapperClass == <span class="literal">null</span>) {</span><br><span class="line"> ErrorContext.instance().resource(resource);</span><br><span class="line"> <span class="keyword">try</span>(<span class="type">InputStream</span> <span class="variable">inputStream</span> <span class="operator">=</span> Resources.getResourceAsStream(resource)) {</span><br><span class="line"> <span class="type">XMLMapperBuilder</span> <span class="variable">mapperParser</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">XMLMapperBuilder</span>(inputStream, configuration, resource, configuration.getSqlFragments());</span><br><span class="line"> <span class="comment">// --------> 我们这没有指定package，所以是走到这</span></span><br><span class="line"> mapperParser.parse();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (resource == <span class="literal">null</span> && url != <span class="literal">null</span> && mapperClass == <span class="literal">null</span>) {</span><br><span class="line"> ErrorContext.instance().resource(url);</span><br><span class="line"> <span class="keyword">try</span>(<span class="type">InputStream</span> <span class="variable">inputStream</span> <span class="operator">=</span> Resources.getUrlAsStream(url)){</span><br><span class="line"> <span class="type">XMLMapperBuilder</span> <span class="variable">mapperParser</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">XMLMapperBuilder</span>(inputStream, configuration, url, configuration.getSqlFragments());</span><br><span class="line"> mapperParser.parse();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (resource == <span class="literal">null</span> && url == <span class="literal">null</span> && mapperClass != <span class="literal">null</span>) {</span><br><span class="line"> Class<?> mapperInterface = Resources.classForName(mapperClass);</span><br><span class="line"> configuration.addMapper(mapperInterface);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">BuilderException</span>(<span class="string">"A mapper element may only specify a url, resource or class, but not more than one."</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>核心就在这个parse()方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">parse</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (!configuration.isResourceLoaded(resource)) {</span><br><span class="line"> <span class="comment">// -------> 然后就是走到这里,配置xml的mapper节点的内容</span></span><br><span class="line"> configurationElement(parser.evalNode(<span class="string">"/mapper"</span>));</span><br><span class="line"> configuration.addLoadedResource(resource);</span><br><span class="line"> bindMapperForNamespace();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> parsePendingResultMaps();</span><br><span class="line"> parsePendingCacheRefs();</span><br><span class="line"> parsePendingStatements();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>具体的处理逻辑</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">configurationElement</span><span class="params">(XNode context)</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">namespace</span> <span class="operator">=</span> context.getStringAttribute(<span class="string">"namespace"</span>);</span><br><span class="line"> <span class="keyword">if</span> (namespace == <span class="literal">null</span> || namespace.isEmpty()) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">BuilderException</span>(<span class="string">"Mapper's namespace cannot be empty"</span>);</span><br><span class="line"> }</span><br><span class="line"> builderAssistant.setCurrentNamespace(namespace);</span><br><span class="line"> cacheRefElement(context.evalNode(<span class="string">"cache-ref"</span>));</span><br><span class="line"> cacheElement(context.evalNode(<span class="string">"cache"</span>));</span><br><span class="line"> parameterMapElement(context.evalNodes(<span class="string">"/mapper/parameterMap"</span>));</span><br><span class="line"> resultMapElements(context.evalNodes(<span class="string">"/mapper/resultMap"</span>));</span><br><span class="line"> sqlElement(context.evalNodes(<span class="string">"/mapper/sql"</span>));</span><br><span class="line"> <span class="comment">// -------> 走到这，从上下文构建statement</span></span><br><span class="line"> buildStatementFromContext(context.evalNodes(<span class="string">"select|insert|update|delete"</span>));</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">BuilderException</span>(<span class="string">"Error parsing Mapper XML. The XML location is '"</span> + resource + <span class="string">"'. Cause: "</span> + e, e);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>具体代码在这，从上下文构建<code>statement</code>，只不过区分了下<code>databaseId</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">buildStatementFromContext</span><span class="params">(List<XNode> list)</span> {</span><br><span class="line"> <span class="keyword">if</span> (configuration.getDatabaseId() != <span class="literal">null</span>) {</span><br><span class="line"> buildStatementFromContext(list, configuration.getDatabaseId());</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// -----> 判断databaseId</span></span><br><span class="line"> buildStatementFromContext(list, <span class="literal">null</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>判断下<code>databaseId</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">buildStatementFromContext</span><span class="params">(List<XNode> list, String requiredDatabaseId)</span> {</span><br><span class="line"> <span class="keyword">for</span> (XNode context : list) {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">XMLStatementBuilder</span> <span class="variable">statementParser</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">XMLStatementBuilder</span>(configuration, builderAssistant, context, requiredDatabaseId);</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// -------> 解析statement节点</span></span><br><span class="line"> statementParser.parseStatementNode();</span><br><span class="line"> } <span class="keyword">catch</span> (IncompleteElementException e) {</span><br><span class="line"> configuration.addIncompleteStatement(statementParser);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>接下来就是真正处理的xml语句内容的，各个节点的信息内容</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">parseStatementNode</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">id</span> <span class="operator">=</span> context.getStringAttribute(<span class="string">"id"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">databaseId</span> <span class="operator">=</span> context.getStringAttribute(<span class="string">"databaseId"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!databaseIdMatchesCurrent(id, databaseId, <span class="built_in">this</span>.requiredDatabaseId)) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">String</span> <span class="variable">nodeName</span> <span class="operator">=</span> context.getNode().getNodeName();</span><br><span class="line"> <span class="type">SqlCommandType</span> <span class="variable">sqlCommandType</span> <span class="operator">=</span> SqlCommandType.valueOf(nodeName.toUpperCase(Locale.ENGLISH));</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">isSelect</span> <span class="operator">=</span> sqlCommandType == SqlCommandType.SELECT;</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">flushCache</span> <span class="operator">=</span> context.getBooleanAttribute(<span class="string">"flushCache"</span>, !isSelect);</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">useCache</span> <span class="operator">=</span> context.getBooleanAttribute(<span class="string">"useCache"</span>, isSelect);</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">resultOrdered</span> <span class="operator">=</span> context.getBooleanAttribute(<span class="string">"resultOrdered"</span>, <span class="literal">false</span>);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Include Fragments before parsing</span></span><br><span class="line"> <span class="type">XMLIncludeTransformer</span> <span class="variable">includeParser</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">XMLIncludeTransformer</span>(configuration, builderAssistant);</span><br><span class="line"> includeParser.applyIncludes(context.getNode());</span><br><span class="line"></span><br><span class="line"> <span class="type">String</span> <span class="variable">parameterType</span> <span class="operator">=</span> context.getStringAttribute(<span class="string">"parameterType"</span>);</span><br><span class="line"> Class<?> parameterTypeClass = resolveClass(parameterType);</span><br><span class="line"></span><br><span class="line"> <span class="type">String</span> <span class="variable">lang</span> <span class="operator">=</span> context.getStringAttribute(<span class="string">"lang"</span>);</span><br><span class="line"> <span class="type">LanguageDriver</span> <span class="variable">langDriver</span> <span class="operator">=</span> getLanguageDriver(lang);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Parse selectKey after includes and remove them.</span></span><br><span class="line"> processSelectKeyNodes(id, parameterTypeClass, langDriver);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Parse the SQL (pre: <selectKey> and <include> were parsed and removed)</span></span><br><span class="line"> KeyGenerator keyGenerator;</span><br><span class="line"> <span class="type">String</span> <span class="variable">keyStatementId</span> <span class="operator">=</span> id + SelectKeyGenerator.SELECT_KEY_SUFFIX;</span><br><span class="line"> keyStatementId = builderAssistant.applyCurrentNamespace(keyStatementId, <span class="literal">true</span>);</span><br><span class="line"> <span class="keyword">if</span> (configuration.hasKeyGenerator(keyStatementId)) {</span><br><span class="line"> keyGenerator = configuration.getKeyGenerator(keyStatementId);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> keyGenerator = context.getBooleanAttribute(<span class="string">"useGeneratedKeys"</span>,</span><br><span class="line"> configuration.isUseGeneratedKeys() && SqlCommandType.INSERT.equals(sqlCommandType))</span><br><span class="line"> ? Jdbc3KeyGenerator.INSTANCE : NoKeyGenerator.INSTANCE;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 语句的主要参数解析</span></span><br><span class="line"> <span class="type">SqlSource</span> <span class="variable">sqlSource</span> <span class="operator">=</span> langDriver.createSqlSource(configuration, context, parameterTypeClass);</span><br><span class="line"> <span class="type">StatementType</span> <span class="variable">statementType</span> <span class="operator">=</span> StatementType.valueOf(context.getStringAttribute(<span class="string">"statementType"</span>, StatementType.PREPARED.toString()));</span><br><span class="line"> <span class="type">Integer</span> <span class="variable">fetchSize</span> <span class="operator">=</span> context.getIntAttribute(<span class="string">"fetchSize"</span>);</span><br><span class="line"> <span class="type">Integer</span> <span class="variable">timeout</span> <span class="operator">=</span> context.getIntAttribute(<span class="string">"timeout"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">parameterMap</span> <span class="operator">=</span> context.getStringAttribute(<span class="string">"parameterMap"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">resultType</span> <span class="operator">=</span> context.getStringAttribute(<span class="string">"resultType"</span>);</span><br><span class="line"> Class<?> resultTypeClass = resolveClass(resultType);</span><br><span class="line"> <span class="type">String</span> <span class="variable">resultMap</span> <span class="operator">=</span> context.getStringAttribute(<span class="string">"resultMap"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">resultSetType</span> <span class="operator">=</span> context.getStringAttribute(<span class="string">"resultSetType"</span>);</span><br><span class="line"> <span class="type">ResultSetType</span> <span class="variable">resultSetTypeEnum</span> <span class="operator">=</span> resolveResultSetType(resultSetType);</span><br><span class="line"> <span class="keyword">if</span> (resultSetTypeEnum == <span class="literal">null</span>) {</span><br><span class="line"> resultSetTypeEnum = configuration.getDefaultResultSetType();</span><br><span class="line"> }</span><br><span class="line"> <span class="type">String</span> <span class="variable">keyProperty</span> <span class="operator">=</span> context.getStringAttribute(<span class="string">"keyProperty"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">keyColumn</span> <span class="operator">=</span> context.getStringAttribute(<span class="string">"keyColumn"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">resultSets</span> <span class="operator">=</span> context.getStringAttribute(<span class="string">"resultSets"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// --------> 添加映射的statement</span></span><br><span class="line"> builderAssistant.addMappedStatement(id, sqlSource, statementType, sqlCommandType,</span><br><span class="line"> fetchSize, timeout, parameterMap, parameterTypeClass, resultMap, resultTypeClass,</span><br><span class="line"> resultSetTypeEnum, flushCache, useCache, resultOrdered,</span><br><span class="line"> keyGenerator, keyProperty, keyColumn, databaseId, langDriver, resultSets);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
<p>添加的逻辑具体可以看下</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> MappedStatement <span class="title function_">addMappedStatement</span><span class="params">(</span></span><br><span class="line"><span class="params"> String id,</span></span><br><span class="line"><span class="params"> SqlSource sqlSource,</span></span><br><span class="line"><span class="params"> StatementType statementType,</span></span><br><span class="line"><span class="params"> SqlCommandType sqlCommandType,</span></span><br><span class="line"><span class="params"> Integer fetchSize,</span></span><br><span class="line"><span class="params"> Integer timeout,</span></span><br><span class="line"><span class="params"> String parameterMap,</span></span><br><span class="line"><span class="params"> Class<?> parameterType,</span></span><br><span class="line"><span class="params"> String resultMap,</span></span><br><span class="line"><span class="params"> Class<?> resultType,</span></span><br><span class="line"><span class="params"> ResultSetType resultSetType,</span></span><br><span class="line"><span class="params"> <span class="type">boolean</span> flushCache,</span></span><br><span class="line"><span class="params"> <span class="type">boolean</span> useCache,</span></span><br><span class="line"><span class="params"> <span class="type">boolean</span> resultOrdered,</span></span><br><span class="line"><span class="params"> KeyGenerator keyGenerator,</span></span><br><span class="line"><span class="params"> String keyProperty,</span></span><br><span class="line"><span class="params"> String keyColumn,</span></span><br><span class="line"><span class="params"> String databaseId,</span></span><br><span class="line"><span class="params"> LanguageDriver lang,</span></span><br><span class="line"><span class="params"> String resultSets)</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (unresolvedCacheRef) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IncompleteElementException</span>(<span class="string">"Cache-ref not yet resolved"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> id = applyCurrentNamespace(id, <span class="literal">false</span>);</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">isSelect</span> <span class="operator">=</span> sqlCommandType == SqlCommandType.SELECT;</span><br><span class="line"></span><br><span class="line"> MappedStatement.<span class="type">Builder</span> <span class="variable">statementBuilder</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">MappedStatement</span>.Builder(configuration, id, sqlSource, sqlCommandType)</span><br><span class="line"> .resource(resource)</span><br><span class="line"> .fetchSize(fetchSize)</span><br><span class="line"> .timeout(timeout)</span><br><span class="line"> .statementType(statementType)</span><br><span class="line"> .keyGenerator(keyGenerator)</span><br><span class="line"> .keyProperty(keyProperty)</span><br><span class="line"> .keyColumn(keyColumn)</span><br><span class="line"> .databaseId(databaseId)</span><br><span class="line"> .lang(lang)</span><br><span class="line"> .resultOrdered(resultOrdered)</span><br><span class="line"> .resultSets(resultSets)</span><br><span class="line"> .resultMaps(getStatementResultMaps(resultMap, resultType, id))</span><br><span class="line"> .resultSetType(resultSetType)</span><br><span class="line"> .flushCacheRequired(valueOrDefault(flushCache, !isSelect))</span><br><span class="line"> .useCache(valueOrDefault(useCache, isSelect))</span><br><span class="line"> .cache(currentCache);</span><br><span class="line"></span><br><span class="line"> <span class="type">ParameterMap</span> <span class="variable">statementParameterMap</span> <span class="operator">=</span> getStatementParameterMap(parameterMap, parameterType, id);</span><br><span class="line"> <span class="keyword">if</span> (statementParameterMap != <span class="literal">null</span>) {</span><br><span class="line"> statementBuilder.parameterMap(statementParameterMap);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">MappedStatement</span> <span class="variable">statement</span> <span class="operator">=</span> statementBuilder.build();</span><br><span class="line"> <span class="comment">// ------> 正好是这里在configuration中添加了映射好的statement</span></span><br><span class="line"> configuration.addMappedStatement(statement);</span><br><span class="line"> <span class="keyword">return</span> statement;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>而里面就是往map里添加</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">addMappedStatement</span><span class="params">(MappedStatement ms)</span> {</span><br><span class="line"> mappedStatements.put(ms.getId(), ms);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<h2 id="获取mapper"><a href="#获取mapper" class="headerlink" title="获取mapper"></a>获取mapper</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">StudentDO</span> <span class="variable">studentDO</span> <span class="operator">=</span> session.selectOne(<span class="string">"com.nicksxs.mybatisdemo.StudentMapper.selectStudent"</span>, <span class="number">1</span>);</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>就是调用了 <code>org.apache.ibatis.session.defaults.DefaultSqlSession#selectOne(java.lang.String, java.lang.Object)</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <T> T <span class="title function_">selectOne</span><span class="params">(String statement, Object parameter)</span> {</span><br><span class="line"> <span class="comment">// Popular vote was to return null on 0 results and throw exception on too many.</span></span><br><span class="line"> List<T> list = <span class="built_in">this</span>.selectList(statement, parameter);</span><br><span class="line"> <span class="keyword">if</span> (list.size() == <span class="number">1</span>) {</span><br><span class="line"> <span class="keyword">return</span> list.get(<span class="number">0</span>);</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (list.size() > <span class="number">1</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">TooManyResultsException</span>(<span class="string">"Expected one result (or null) to be returned by selectOne(), but found: "</span> + list.size());</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>调用实际的实现方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <E> List<E> <span class="title function_">selectList</span><span class="params">(String statement, Object parameter)</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.selectList(statement, parameter, RowBounds.DEFAULT);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这里还有一层</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <E> List<E> <span class="title function_">selectList</span><span class="params">(String statement, Object parameter, RowBounds rowBounds)</span> {</span><br><span class="line"> <span class="keyword">return</span> selectList(statement, parameter, rowBounds, Executor.NO_RESULT_HANDLER);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
<p>根本的就是从configuration里获取了mappedStatement</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <E> List<E> <span class="title function_">selectList</span><span class="params">(String statement, Object parameter, RowBounds rowBounds, ResultHandler handler)</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 这里进行了获取</span></span><br><span class="line"> <span class="type">MappedStatement</span> <span class="variable">ms</span> <span class="operator">=</span> configuration.getMappedStatement(statement);</span><br><span class="line"> <span class="keyword">return</span> executor.query(ms, wrapCollection(parameter), rowBounds, handler);</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> ExceptionFactory.wrapException(<span class="string">"Error querying database. Cause: "</span> + e, e);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> ErrorContext.instance().reset();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Mybatis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Mysql</tag>
|
|
<tag>Mybatis</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>mybatis系列-connection连接池解析</title>
|
|
<url>/2023/02/19/mybatis%E7%B3%BB%E5%88%97-connection%E8%BF%9E%E6%8E%A5%E6%B1%A0%E8%A7%A3%E6%9E%90/</url>
|
|
<content><![CDATA[<p>连接池主要是两个逻辑，首先是获取连接的逻辑，结合代码来讲一讲</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> PooledConnection <span class="title function_">popConnection</span><span class="params">(String username, String password)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">countedWait</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="type">PooledConnection</span> <span class="variable">conn</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="type">long</span> <span class="variable">t</span> <span class="operator">=</span> System.currentTimeMillis();</span><br><span class="line"> <span class="type">int</span> <span class="variable">localBadConnectionCount</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 循环获取连接</span></span><br><span class="line"> <span class="keyword">while</span> (conn == <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// 加锁</span></span><br><span class="line"> lock.lock();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 如果闲置的连接列表不为空</span></span><br><span class="line"> <span class="keyword">if</span> (!state.idleConnections.isEmpty()) {</span><br><span class="line"> <span class="comment">// Pool has available connection</span></span><br><span class="line"> <span class="comment">// 连接池有可用的连接</span></span><br><span class="line"> conn = state.idleConnections.remove(<span class="number">0</span>);</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"Checked out connection "</span> + conn.getRealHashCode() + <span class="string">" from pool."</span>);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// Pool does not have available connection</span></span><br><span class="line"> <span class="comment">// 进入这个分支表示没有空闲连接，但是活跃连接数还没达到最大活跃连接数上限，那么这时候就可以创建一个新连接</span></span><br><span class="line"> <span class="keyword">if</span> (state.activeConnections.size() < poolMaximumActiveConnections) {</span><br><span class="line"> <span class="comment">// Can create new connection</span></span><br><span class="line"> <span class="comment">// 这里创建连接我们之前讲过，</span></span><br><span class="line"> conn = <span class="keyword">new</span> <span class="title class_">PooledConnection</span>(dataSource.getConnection(), <span class="built_in">this</span>);</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"Created connection "</span> + conn.getRealHashCode() + <span class="string">"."</span>);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// Cannot create new connection</span></span><br><span class="line"> <span class="comment">// 进到这个分支了就表示没法创建新连接了，那么怎么办呢，这里引入了一个 poolMaximumCheckoutTime，这代表了我去控制连接一次被使用的最长时间，如果超过这个时间了，我就要去关闭失效它</span></span><br><span class="line"> <span class="type">PooledConnection</span> <span class="variable">oldestActiveConnection</span> <span class="operator">=</span> state.activeConnections.get(<span class="number">0</span>);</span><br><span class="line"> <span class="type">long</span> <span class="variable">longestCheckoutTime</span> <span class="operator">=</span> oldestActiveConnection.getCheckoutTime();</span><br><span class="line"> <span class="keyword">if</span> (longestCheckoutTime > poolMaximumCheckoutTime) {</span><br><span class="line"> <span class="comment">// Can claim overdue connection</span></span><br><span class="line"> <span class="comment">// 所有超时连接从池中被借出的次数+1</span></span><br><span class="line"> state.claimedOverdueConnectionCount++;</span><br><span class="line"> <span class="comment">// 所有超时连接从池中被借出并归还的时间总和 + 当前连接借出时间</span></span><br><span class="line"> state.accumulatedCheckoutTimeOfOverdueConnections += longestCheckoutTime;</span><br><span class="line"> <span class="comment">// 所有连接从池中被借出并归还的时间总和 + 当前连接借出时间</span></span><br><span class="line"> state.accumulatedCheckoutTime += longestCheckoutTime;</span><br><span class="line"> <span class="comment">// 从活跃连接数中移除此连接</span></span><br><span class="line"> state.activeConnections.remove(oldestActiveConnection);</span><br><span class="line"> <span class="comment">// 如果该连接不是自动提交的，则尝试回滚</span></span><br><span class="line"> <span class="keyword">if</span> (!oldestActiveConnection.getRealConnection().getAutoCommit()) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> oldestActiveConnection.getRealConnection().rollback();</span><br><span class="line"> } <span class="keyword">catch</span> (SQLException e) {</span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> Just log a message for debug and continue to execute the following</span></span><br><span class="line"><span class="comment"> statement like nothing happened.</span></span><br><span class="line"><span class="comment"> Wrap the bad connection with a new PooledConnection, this will help</span></span><br><span class="line"><span class="comment"> to not interrupt current executing thread and give current thread a</span></span><br><span class="line"><span class="comment"> chance to join the next competition for another valid/good database</span></span><br><span class="line"><span class="comment"> connection. At the end of this loop, bad {@link @conn} will be set as null.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> log.debug(<span class="string">"Bad connection. Could not roll back"</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 用此连接的真实连接再创建一个连接，并设置时间</span></span><br><span class="line"> conn = <span class="keyword">new</span> <span class="title class_">PooledConnection</span>(oldestActiveConnection.getRealConnection(), <span class="built_in">this</span>);</span><br><span class="line"> conn.setCreatedTimestamp(oldestActiveConnection.getCreatedTimestamp());</span><br><span class="line"> conn.setLastUsedTimestamp(oldestActiveConnection.getLastUsedTimestamp());</span><br><span class="line"> oldestActiveConnection.invalidate();</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"Claimed overdue connection "</span> + conn.getRealHashCode() + <span class="string">"."</span>);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// Must wait</span></span><br><span class="line"> <span class="comment">// 这样还是获取不到连接就只能等待了</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 标记状态，然后把等待计数+1</span></span><br><span class="line"> <span class="keyword">if</span> (!countedWait) {</span><br><span class="line"> state.hadToWaitCount++;</span><br><span class="line"> countedWait = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"Waiting as long as "</span> + poolTimeToWait + <span class="string">" milliseconds for connection."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="type">long</span> <span class="variable">wt</span> <span class="operator">=</span> System.currentTimeMillis();</span><br><span class="line"> <span class="comment">// 等待 poolTimeToWait 时间</span></span><br><span class="line"> condition.await(poolTimeToWait, TimeUnit.MILLISECONDS);</span><br><span class="line"> <span class="comment">// 记录等待时间</span></span><br><span class="line"> state.accumulatedWaitTime += System.currentTimeMillis() - wt;</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> <span class="comment">// set interrupt flag</span></span><br><span class="line"> Thread.currentThread().interrupt();</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 如果连接不为空</span></span><br><span class="line"> <span class="keyword">if</span> (conn != <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// ping to server and check the connection is valid or not</span></span><br><span class="line"> <span class="comment">// 判断是否有效</span></span><br><span class="line"> <span class="keyword">if</span> (conn.isValid()) {</span><br><span class="line"> <span class="keyword">if</span> (!conn.getRealConnection().getAutoCommit()) {</span><br><span class="line"> <span class="comment">// 回滚未提交的</span></span><br><span class="line"> conn.getRealConnection().rollback();</span><br><span class="line"> }</span><br><span class="line"> conn.setConnectionTypeCode(assembleConnectionTypeCode(dataSource.getUrl(), username, password));</span><br><span class="line"> <span class="comment">// 设置时间</span></span><br><span class="line"> conn.setCheckoutTimestamp(System.currentTimeMillis());</span><br><span class="line"> conn.setLastUsedTimestamp(System.currentTimeMillis());</span><br><span class="line"> <span class="comment">// 添加进活跃连接</span></span><br><span class="line"> state.activeConnections.add(conn);</span><br><span class="line"> state.requestCount++;</span><br><span class="line"> state.accumulatedRequestTime += System.currentTimeMillis() - t;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"A bad connection ("</span> + conn.getRealHashCode() + <span class="string">") was returned from the pool, getting another connection."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 连接无效，坏连接+1</span></span><br><span class="line"> state.badConnectionCount++;</span><br><span class="line"> localBadConnectionCount++;</span><br><span class="line"> conn = <span class="literal">null</span>;</span><br><span class="line"> <span class="comment">// 如果坏连接已经超过了容忍上限，就抛异常</span></span><br><span class="line"> <span class="keyword">if</span> (localBadConnectionCount > (poolMaximumIdleConnections + poolMaximumLocalBadConnectionTolerance)) {</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"PooledDataSource: Could not get a good connection to the database."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">SQLException</span>(<span class="string">"PooledDataSource: Could not get a good connection to the database."</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="comment">// 释放锁</span></span><br><span class="line"> lock.unlock();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (conn == <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// 连接仍为空</span></span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"PooledDataSource: Unknown severe error condition. The connection pool returned a null connection."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 抛出异常</span></span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">SQLException</span>(<span class="string">"PooledDataSource: Unknown severe error condition. The connection pool returned a null connection."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// fanhui </span></span><br><span class="line"> <span class="keyword">return</span> conn;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后是还回连接</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">pushConnection</span><span class="params">(PooledConnection conn)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="comment">// 加锁</span></span><br><span class="line"> lock.lock();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 从活跃连接中移除当前连接</span></span><br><span class="line"> state.activeConnections.remove(conn);</span><br><span class="line"> <span class="keyword">if</span> (conn.isValid()) {</span><br><span class="line"> <span class="comment">// 当前的空闲连接数小于连接池中允许的最大空闲连接数</span></span><br><span class="line"> <span class="keyword">if</span> (state.idleConnections.size() < poolMaximumIdleConnections && conn.getConnectionTypeCode() == expectedConnectionTypeCode) {</span><br><span class="line"> <span class="comment">// 记录借出时间</span></span><br><span class="line"> state.accumulatedCheckoutTime += conn.getCheckoutTime();</span><br><span class="line"> <span class="keyword">if</span> (!conn.getRealConnection().getAutoCommit()) {</span><br><span class="line"> <span class="comment">// 同样是做回滚</span></span><br><span class="line"> conn.getRealConnection().rollback();</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 新建一个连接</span></span><br><span class="line"> <span class="type">PooledConnection</span> <span class="variable">newConn</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">PooledConnection</span>(conn.getRealConnection(), <span class="built_in">this</span>);</span><br><span class="line"> <span class="comment">// 加入到空闲连接列表中</span></span><br><span class="line"> state.idleConnections.add(newConn);</span><br><span class="line"> newConn.setCreatedTimestamp(conn.getCreatedTimestamp());</span><br><span class="line"> newConn.setLastUsedTimestamp(conn.getLastUsedTimestamp());</span><br><span class="line"> <span class="comment">// 原连接失效</span></span><br><span class="line"> conn.invalidate();</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"Returned connection "</span> + newConn.getRealHashCode() + <span class="string">" to pool."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 提醒前面等待的</span></span><br><span class="line"> condition.signal();</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// 上面是相同的，就是这里是空闲连接数已经超过上限</span></span><br><span class="line"> state.accumulatedCheckoutTime += conn.getCheckoutTime();</span><br><span class="line"> <span class="keyword">if</span> (!conn.getRealConnection().getAutoCommit()) {</span><br><span class="line"> conn.getRealConnection().rollback();</span><br><span class="line"> }</span><br><span class="line"> conn.getRealConnection().close();</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"Closed connection "</span> + conn.getRealHashCode() + <span class="string">"."</span>);</span><br><span class="line"> }</span><br><span class="line"> conn.invalidate();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"A bad connection ("</span> + conn.getRealHashCode() + <span class="string">") attempted to return to the pool, discarding connection."</span>);</span><br><span class="line"> }</span><br><span class="line"> state.badConnectionCount++;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> lock.unlock();</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Mybatis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Mysql</tag>
|
|
<tag>Mybatis</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>mybatis系列-dataSource解析</title>
|
|
<url>/2023/01/08/mybatis%E7%B3%BB%E5%88%97-dataSource%E8%A7%A3%E6%9E%90/</url>
|
|
<content><![CDATA[<p>DataSource 作为数据库查询的最重要的数据源，在 mybatis 中也展开来说下<br>首先是解析的过程</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">SqlSessionFactory</span> <span class="variable">sqlSessionFactory</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SqlSessionFactoryBuilder</span>().build(inputStream);</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>在构建 SqlSessionFactory 也就是 DefaultSqlSessionFactory 的时候，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> SqlSessionFactory <span class="title function_">build</span><span class="params">(InputStream inputStream)</span> {</span><br><span class="line"> <span class="keyword">return</span> build(inputStream, <span class="literal">null</span>, <span class="literal">null</span>);</span><br><span class="line"> }</span><br><span class="line"><span class="keyword">public</span> SqlSessionFactory <span class="title function_">build</span><span class="params">(InputStream inputStream, String environment, Properties properties)</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">XMLConfigBuilder</span> <span class="variable">parser</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">XMLConfigBuilder</span>(inputStream, environment, properties);</span><br><span class="line"> <span class="keyword">return</span> build(parser.parse());</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> ExceptionFactory.wrapException(<span class="string">"Error building SqlSession."</span>, e);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> ErrorContext.instance().reset();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (inputStream != <span class="literal">null</span>) {</span><br><span class="line"> inputStream.close();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (IOException e) {</span><br><span class="line"> <span class="comment">// Intentionally ignore. Prefer previous error.</span></span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>前面也说过，就是解析 <code>mybatis-config.xml</code> 成 <code>Configuration</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Configuration <span class="title function_">parse</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (parsed) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">BuilderException</span>(<span class="string">"Each XMLConfigBuilder can only be used once."</span>);</span><br><span class="line"> }</span><br><span class="line"> parsed = <span class="literal">true</span>;</span><br><span class="line"> parseConfiguration(parser.evalNode(<span class="string">"/configuration"</span>));</span><br><span class="line"> <span class="keyword">return</span> configuration;</span><br><span class="line">}</span><br><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">parseConfiguration</span><span class="params">(XNode root)</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// issue #117 read properties first</span></span><br><span class="line"> propertiesElement(root.evalNode(<span class="string">"properties"</span>));</span><br><span class="line"> <span class="type">Properties</span> <span class="variable">settings</span> <span class="operator">=</span> settingsAsProperties(root.evalNode(<span class="string">"settings"</span>));</span><br><span class="line"> loadCustomVfs(settings);</span><br><span class="line"> loadCustomLogImpl(settings);</span><br><span class="line"> typeAliasesElement(root.evalNode(<span class="string">"typeAliases"</span>));</span><br><span class="line"> pluginElement(root.evalNode(<span class="string">"plugins"</span>));</span><br><span class="line"> objectFactoryElement(root.evalNode(<span class="string">"objectFactory"</span>));</span><br><span class="line"> objectWrapperFactoryElement(root.evalNode(<span class="string">"objectWrapperFactory"</span>));</span><br><span class="line"> reflectorFactoryElement(root.evalNode(<span class="string">"reflectorFactory"</span>));</span><br><span class="line"> settingsElement(settings);</span><br><span class="line"> <span class="comment">// read it after objectFactory and objectWrapperFactory issue #631</span></span><br><span class="line"> <span class="comment">// -------------> 是在这里解析了DataSource</span></span><br><span class="line"> environmentsElement(root.evalNode(<span class="string">"environments"</span>));</span><br><span class="line"> databaseIdProviderElement(root.evalNode(<span class="string">"databaseIdProvider"</span>));</span><br><span class="line"> typeHandlerElement(root.evalNode(<span class="string">"typeHandlers"</span>));</span><br><span class="line"> mapperElement(root.evalNode(<span class="string">"mappers"</span>));</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">BuilderException</span>(<span class="string">"Error parsing SQL Mapper Configuration. Cause: "</span> + e, e);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>环境解析了这一块的内容</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">environments</span> <span class="attr">default</span>=<span class="string">"development"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">environment</span> <span class="attr">id</span>=<span class="string">"development"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">transactionManager</span> <span class="attr">type</span>=<span class="string">"JDBC"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">dataSource</span> <span class="attr">type</span>=<span class="string">"POOLED"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"driver"</span> <span class="attr">value</span>=<span class="string">"${driver}"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"url"</span> <span class="attr">value</span>=<span class="string">"${url}"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"username"</span> <span class="attr">value</span>=<span class="string">"${username}"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"password"</span> <span class="attr">value</span>=<span class="string">"${password}"</span>/></span></span><br><span class="line"> <span class="tag"></<span class="name">dataSource</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">environment</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">environments</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>解析也是自上而下的，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">environmentsElement</span><span class="params">(XNode context)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="keyword">if</span> (context != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">if</span> (environment == <span class="literal">null</span>) {</span><br><span class="line"> environment = context.getStringAttribute(<span class="string">"default"</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">for</span> (XNode child : context.getChildren()) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">id</span> <span class="operator">=</span> child.getStringAttribute(<span class="string">"id"</span>);</span><br><span class="line"> <span class="keyword">if</span> (isSpecifiedEnvironment(id)) {</span><br><span class="line"> <span class="type">TransactionFactory</span> <span class="variable">txFactory</span> <span class="operator">=</span> transactionManagerElement(child.evalNode(<span class="string">"transactionManager"</span>));</span><br><span class="line"> <span class="type">DataSourceFactory</span> <span class="variable">dsFactory</span> <span class="operator">=</span> dataSourceElement(child.evalNode(<span class="string">"dataSource"</span>));</span><br><span class="line"> <span class="type">DataSource</span> <span class="variable">dataSource</span> <span class="operator">=</span> dsFactory.getDataSource();</span><br><span class="line"> Environment.<span class="type">Builder</span> <span class="variable">environmentBuilder</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Environment</span>.Builder(id)</span><br><span class="line"> .transactionFactory(txFactory)</span><br><span class="line"> .dataSource(dataSource);</span><br><span class="line"> configuration.setEnvironment(environmentBuilder.build());</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>前面第一步是解析事务管理器元素</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> TransactionFactory <span class="title function_">transactionManagerElement</span><span class="params">(XNode context)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="keyword">if</span> (context != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">type</span> <span class="operator">=</span> context.getStringAttribute(<span class="string">"type"</span>);</span><br><span class="line"> <span class="type">Properties</span> <span class="variable">props</span> <span class="operator">=</span> context.getChildrenAsProperties();</span><br><span class="line"> <span class="type">TransactionFactory</span> <span class="variable">factory</span> <span class="operator">=</span> (TransactionFactory) resolveClass(type).getDeclaredConstructor().newInstance();</span><br><span class="line"> factory.setProperties(props);</span><br><span class="line"> <span class="keyword">return</span> factory;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">BuilderException</span>(<span class="string">"Environment declaration requires a TransactionFactory."</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>而这里的 <code>resolveClass</code> 其实就使用了上一篇的 <code>typeAliases</code> 系统，这里是使用了 <code>JdbcTransactionFactory</code> 作为事务管理器，<br>后面的就是 <code>DataSourceFactory</code> 的创建也是 <code>DataSource</code> 的创建</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> DataSourceFactory <span class="title function_">dataSourceElement</span><span class="params">(XNode context)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="keyword">if</span> (context != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">type</span> <span class="operator">=</span> context.getStringAttribute(<span class="string">"type"</span>);</span><br><span class="line"> <span class="type">Properties</span> <span class="variable">props</span> <span class="operator">=</span> context.getChildrenAsProperties();</span><br><span class="line"> <span class="type">DataSourceFactory</span> <span class="variable">factory</span> <span class="operator">=</span> (DataSourceFactory) resolveClass(type).getDeclaredConstructor().newInstance();</span><br><span class="line"> factory.setProperties(props);</span><br><span class="line"> <span class="keyword">return</span> factory;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">BuilderException</span>(<span class="string">"Environment declaration requires a DataSourceFactory."</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>因为在config文件中设置了Pooled，所以对应创建的就是 <code>PooledDataSourceFactory</code><br>但是这里其实有个比较需要注意的，mybatis 这里的其实是继承了 <code>UnpooledDataSourceFactory</code><br>将基础方法都放在了 <code>UnpooledDataSourceFactory</code> 中</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">PooledDataSourceFactory</span> <span class="keyword">extends</span> <span class="title class_">UnpooledDataSourceFactory</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">PooledDataSourceFactory</span><span class="params">()</span> {</span><br><span class="line"> <span class="built_in">this</span>.dataSource = <span class="keyword">new</span> <span class="title class_">PooledDataSource</span>();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里只保留了在构造方法里创建 <code>DataSource</code><br>而这个 <code>PooledDataSource</code> 虽然没有直接继承 <code>UnpooledDataSource</code>，但其实<br>在构造方法里也是</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">PooledDataSource</span><span class="params">()</span> {</span><br><span class="line"> dataSource = <span class="keyword">new</span> <span class="title class_">UnpooledDataSource</span>();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>至于为什么这么做呢应该也是考虑到能比较多的复用代码，因为 <code>Pooled</code> 其实跟 <code>Unpooled</code> 最重要的差别就在于是不是每次都重开连接<br>使用连接池能够让应用在有大量查询的时候不用反复创建连接，省去了建联的网络等开销，<code>Unpooled</code>就是完成与数据库的连接，并可以获取该连接<br>主要的代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> Connection <span class="title function_">getConnection</span><span class="params">()</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="keyword">return</span> doGetConnection(username, password);</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> Connection <span class="title function_">getConnection</span><span class="params">(String username, String password)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="keyword">return</span> doGetConnection(username, password);</span><br><span class="line">}</span><br><span class="line"><span class="keyword">private</span> Connection <span class="title function_">doGetConnection</span><span class="params">(String username, String password)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="type">Properties</span> <span class="variable">props</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Properties</span>();</span><br><span class="line"> <span class="keyword">if</span> (driverProperties != <span class="literal">null</span>) {</span><br><span class="line"> props.putAll(driverProperties);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (username != <span class="literal">null</span>) {</span><br><span class="line"> props.setProperty(<span class="string">"user"</span>, username);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (password != <span class="literal">null</span>) {</span><br><span class="line"> props.setProperty(<span class="string">"password"</span>, password);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> doGetConnection(props);</span><br><span class="line">}</span><br><span class="line"><span class="keyword">private</span> Connection <span class="title function_">doGetConnection</span><span class="params">(Properties properties)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> initializeDriver();</span><br><span class="line"> <span class="type">Connection</span> <span class="variable">connection</span> <span class="operator">=</span> DriverManager.getConnection(url, properties);</span><br><span class="line"> configureConnection(connection);</span><br><span class="line"> <span class="keyword">return</span> connection;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>而对于Pooled就会处理池化的逻辑</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> PooledConnection <span class="title function_">popConnection</span><span class="params">(String username, String password)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">countedWait</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="type">PooledConnection</span> <span class="variable">conn</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="type">long</span> <span class="variable">t</span> <span class="operator">=</span> System.currentTimeMillis();</span><br><span class="line"> <span class="type">int</span> <span class="variable">localBadConnectionCount</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">while</span> (conn == <span class="literal">null</span>) {</span><br><span class="line"> lock.lock();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (!state.idleConnections.isEmpty()) {</span><br><span class="line"> <span class="comment">// Pool has available connection</span></span><br><span class="line"> conn = state.idleConnections.remove(<span class="number">0</span>);</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"Checked out connection "</span> + conn.getRealHashCode() + <span class="string">" from pool."</span>);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// Pool does not have available connection</span></span><br><span class="line"> <span class="keyword">if</span> (state.activeConnections.size() < poolMaximumActiveConnections) {</span><br><span class="line"> <span class="comment">// Can create new connection</span></span><br><span class="line"> conn = <span class="keyword">new</span> <span class="title class_">PooledConnection</span>(dataSource.getConnection(), <span class="built_in">this</span>);</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"Created connection "</span> + conn.getRealHashCode() + <span class="string">"."</span>);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// Cannot create new connection</span></span><br><span class="line"> <span class="type">PooledConnection</span> <span class="variable">oldestActiveConnection</span> <span class="operator">=</span> state.activeConnections.get(<span class="number">0</span>);</span><br><span class="line"> <span class="type">long</span> <span class="variable">longestCheckoutTime</span> <span class="operator">=</span> oldestActiveConnection.getCheckoutTime();</span><br><span class="line"> <span class="keyword">if</span> (longestCheckoutTime > poolMaximumCheckoutTime) {</span><br><span class="line"> <span class="comment">// Can claim overdue connection</span></span><br><span class="line"> state.claimedOverdueConnectionCount++;</span><br><span class="line"> state.accumulatedCheckoutTimeOfOverdueConnections += longestCheckoutTime;</span><br><span class="line"> state.accumulatedCheckoutTime += longestCheckoutTime;</span><br><span class="line"> state.activeConnections.remove(oldestActiveConnection);</span><br><span class="line"> <span class="keyword">if</span> (!oldestActiveConnection.getRealConnection().getAutoCommit()) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> oldestActiveConnection.getRealConnection().rollback();</span><br><span class="line"> } <span class="keyword">catch</span> (SQLException e) {</span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> Just log a message for debug and continue to execute the following</span></span><br><span class="line"><span class="comment"> statement like nothing happened.</span></span><br><span class="line"><span class="comment"> Wrap the bad connection with a new PooledConnection, this will help</span></span><br><span class="line"><span class="comment"> to not interrupt current executing thread and give current thread a</span></span><br><span class="line"><span class="comment"> chance to join the next competition for another valid/good database</span></span><br><span class="line"><span class="comment"> connection. At the end of this loop, bad {@link @conn} will be set as null.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> log.debug(<span class="string">"Bad connection. Could not roll back"</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> conn = <span class="keyword">new</span> <span class="title class_">PooledConnection</span>(oldestActiveConnection.getRealConnection(), <span class="built_in">this</span>);</span><br><span class="line"> conn.setCreatedTimestamp(oldestActiveConnection.getCreatedTimestamp());</span><br><span class="line"> conn.setLastUsedTimestamp(oldestActiveConnection.getLastUsedTimestamp());</span><br><span class="line"> oldestActiveConnection.invalidate();</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"Claimed overdue connection "</span> + conn.getRealHashCode() + <span class="string">"."</span>);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// Must wait</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (!countedWait) {</span><br><span class="line"> state.hadToWaitCount++;</span><br><span class="line"> countedWait = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"Waiting as long as "</span> + poolTimeToWait + <span class="string">" milliseconds for connection."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="type">long</span> <span class="variable">wt</span> <span class="operator">=</span> System.currentTimeMillis();</span><br><span class="line"> condition.await(poolTimeToWait, TimeUnit.MILLISECONDS);</span><br><span class="line"> state.accumulatedWaitTime += System.currentTimeMillis() - wt;</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> <span class="comment">// set interrupt flag</span></span><br><span class="line"> Thread.currentThread().interrupt();</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (conn != <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// ping to server and check the connection is valid or not</span></span><br><span class="line"> <span class="keyword">if</span> (conn.isValid()) {</span><br><span class="line"> <span class="keyword">if</span> (!conn.getRealConnection().getAutoCommit()) {</span><br><span class="line"> conn.getRealConnection().rollback();</span><br><span class="line"> }</span><br><span class="line"> conn.setConnectionTypeCode(assembleConnectionTypeCode(dataSource.getUrl(), username, password));</span><br><span class="line"> conn.setCheckoutTimestamp(System.currentTimeMillis());</span><br><span class="line"> conn.setLastUsedTimestamp(System.currentTimeMillis());</span><br><span class="line"> state.activeConnections.add(conn);</span><br><span class="line"> state.requestCount++;</span><br><span class="line"> state.accumulatedRequestTime += System.currentTimeMillis() - t;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"A bad connection ("</span> + conn.getRealHashCode() + <span class="string">") was returned from the pool, getting another connection."</span>);</span><br><span class="line"> }</span><br><span class="line"> state.badConnectionCount++;</span><br><span class="line"> localBadConnectionCount++;</span><br><span class="line"> conn = <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">if</span> (localBadConnectionCount > (poolMaximumIdleConnections + poolMaximumLocalBadConnectionTolerance)) {</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"PooledDataSource: Could not get a good connection to the database."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">SQLException</span>(<span class="string">"PooledDataSource: Could not get a good connection to the database."</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> lock.unlock();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (conn == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"PooledDataSource: Unknown severe error condition. The connection pool returned a null connection."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">SQLException</span>(<span class="string">"PooledDataSource: Unknown severe error condition. The connection pool returned a null connection."</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> conn;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>它的入口不是个get方法，而是pop，从含义来来讲就不一样<br><code>org.apache.ibatis.datasource.pooled.PooledDataSource#getConnection()</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> Connection <span class="title function_">getConnection</span><span class="params">()</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="keyword">return</span> popConnection(dataSource.getUsername(), dataSource.getPassword()).getProxyConnection();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>对于具体怎么获取连接我们可以下一篇具体讲下</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Mybatis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Mysql</tag>
|
|
<tag>Mybatis</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>mybatis系列-foreach 解析</title>
|
|
<url>/2023/06/11/mybatis%E7%B3%BB%E5%88%97-foreach-%E8%A7%A3%E6%9E%90/</url>
|
|
<content><![CDATA[<p>在 org.apache.ibatis.builder.xml.XMLConfigBuilder#parseConfiguration 中进行配置解析，其中这一行就是解析 mappers</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">mapperElement(root.evalNode(<span class="string">"mappers"</span>));</span><br></pre></td></tr></table></figure>
|
|
<p>具体的代码会执行到这</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">mapperElement</span><span class="params">(XNode parent)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="keyword">if</span> (parent != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">for</span> (XNode child : parent.getChildren()) {</span><br><span class="line"> <span class="keyword">if</span> (<span class="string">"package"</span>.equals(child.getName())) {</span><br><span class="line"> <span class="comment">// 这里解析的不是 package</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">mapperPackage</span> <span class="operator">=</span> child.getStringAttribute(<span class="string">"name"</span>);</span><br><span class="line"> configuration.addMappers(mapperPackage);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// 根据 resource 和 url 还有 mapperClass 判断</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">resource</span> <span class="operator">=</span> child.getStringAttribute(<span class="string">"resource"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">url</span> <span class="operator">=</span> child.getStringAttribute(<span class="string">"url"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">mapperClass</span> <span class="operator">=</span> child.getStringAttribute(<span class="string">"class"</span>);</span><br><span class="line"> <span class="comment">// resource 不为空其他为空的情况，就开始将 resource 读成输入流</span></span><br><span class="line"> <span class="keyword">if</span> (resource != <span class="literal">null</span> && url == <span class="literal">null</span> && mapperClass == <span class="literal">null</span>) {</span><br><span class="line"> ErrorContext.instance().resource(resource);</span><br><span class="line"> <span class="keyword">try</span>(<span class="type">InputStream</span> <span class="variable">inputStream</span> <span class="operator">=</span> Resources.getResourceAsStream(resource)) {</span><br><span class="line"> <span class="comment">// 初始化 XMLMapperBuilder 来解析 mapper</span></span><br><span class="line"> <span class="type">XMLMapperBuilder</span> <span class="variable">mapperParser</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">XMLMapperBuilder</span>(inputStream, configuration, resource, configuration.getSqlFragments());</span><br><span class="line"> mapperParser.parse();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后再是 parse 过程</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">parse</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (!configuration.isResourceLoaded(resource)) {</span><br><span class="line"> <span class="comment">// 解析 mapper 节点，也就是下图中的mapper</span></span><br><span class="line"> configurationElement(parser.evalNode(<span class="string">"/mapper"</span>));</span><br><span class="line"> configuration.addLoadedResource(resource);</span><br><span class="line"> bindMapperForNamespace();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> parsePendingResultMaps();</span><br><span class="line"> parsePendingCacheRefs();</span><br><span class="line"> parsePendingStatements();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/blog/6wAWb4.png" alt="image"></p>
|
|
<p>继续往下走</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">configurationElement</span><span class="params">(XNode context)</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">namespace</span> <span class="operator">=</span> context.getStringAttribute(<span class="string">"namespace"</span>);</span><br><span class="line"> <span class="keyword">if</span> (namespace == <span class="literal">null</span> || namespace.isEmpty()) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">BuilderException</span>(<span class="string">"Mapper's namespace cannot be empty"</span>);</span><br><span class="line"> }</span><br><span class="line"> builderAssistant.setCurrentNamespace(namespace);</span><br><span class="line"> <span class="comment">// 处理cache 和 cache 应用</span></span><br><span class="line"> cacheRefElement(context.evalNode(<span class="string">"cache-ref"</span>));</span><br><span class="line"> cacheElement(context.evalNode(<span class="string">"cache"</span>));</span><br><span class="line"> parameterMapElement(context.evalNodes(<span class="string">"/mapper/parameterMap"</span>));</span><br><span class="line"> resultMapElements(context.evalNodes(<span class="string">"/mapper/resultMap"</span>));</span><br><span class="line"> sqlElement(context.evalNodes(<span class="string">"/mapper/sql"</span>));</span><br><span class="line"> <span class="comment">// 因为我们是个 sql 查询，所以具体逻辑是在这里面</span></span><br><span class="line"> buildStatementFromContext(context.evalNodes(<span class="string">"select|insert|update|delete"</span>));</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">BuilderException</span>(<span class="string">"Error parsing Mapper XML. The XML location is '"</span> + resource + <span class="string">"'. Cause: "</span> + e, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后是</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">buildStatementFromContext</span><span class="params">(List<XNode> list)</span> {</span><br><span class="line"> <span class="keyword">if</span> (configuration.getDatabaseId() != <span class="literal">null</span>) {</span><br><span class="line"> buildStatementFromContext(list, configuration.getDatabaseId());</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 然后没有 databaseId 就走到这</span></span><br><span class="line"> buildStatementFromContext(list, <span class="literal">null</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>继续</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">buildStatementFromContext</span><span class="params">(List<XNode> list, String requiredDatabaseId)</span> {</span><br><span class="line"> <span class="keyword">for</span> (XNode context : list) {</span><br><span class="line"> <span class="comment">// 创建语句解析器</span></span><br><span class="line"> <span class="keyword">final</span> <span class="type">XMLStatementBuilder</span> <span class="variable">statementParser</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">XMLStatementBuilder</span>(configuration, builderAssistant, context, requiredDatabaseId);</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 解析节点</span></span><br><span class="line"> statementParser.parseStatementNode();</span><br><span class="line"> } <span class="keyword">catch</span> (IncompleteElementException e) {</span><br><span class="line"> configuration.addIncompleteStatement(statementParser);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这个代码比较长，做下简略，只保留相关代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">parseStatementNode</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">id</span> <span class="operator">=</span> context.getStringAttribute(<span class="string">"id"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">databaseId</span> <span class="operator">=</span> context.getStringAttribute(<span class="string">"databaseId"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!databaseIdMatchesCurrent(id, databaseId, <span class="built_in">this</span>.requiredDatabaseId)) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">String</span> <span class="variable">nodeName</span> <span class="operator">=</span> context.getNode().getNodeName();</span><br><span class="line"> <span class="type">SqlCommandType</span> <span class="variable">sqlCommandType</span> <span class="operator">=</span> SqlCommandType.valueOf(nodeName.toUpperCase(Locale.ENGLISH));</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">isSelect</span> <span class="operator">=</span> sqlCommandType == SqlCommandType.SELECT;</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">flushCache</span> <span class="operator">=</span> context.getBooleanAttribute(<span class="string">"flushCache"</span>, !isSelect);</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">useCache</span> <span class="operator">=</span> context.getBooleanAttribute(<span class="string">"useCache"</span>, isSelect);</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">resultOrdered</span> <span class="operator">=</span> context.getBooleanAttribute(<span class="string">"resultOrdered"</span>, <span class="literal">false</span>);</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">// 简略前后代码，主要看这里，创建 sqlSource</span></span><br><span class="line"></span><br><span class="line"> <span class="type">SqlSource</span> <span class="variable">sqlSource</span> <span class="operator">=</span> langDriver.createSqlSource(configuration, context, parameterTypeClass);</span><br><span class="line"> </span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
<p>然后根据 LanguageDriver，我们这用的 XMLLanguageDriver，先是初始化</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> SqlSource <span class="title function_">createSqlSource</span><span class="params">(Configuration configuration, XNode script, Class<?> parameterType)</span> {</span><br><span class="line"> <span class="type">XMLScriptBuilder</span> <span class="variable">builder</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">XMLScriptBuilder</span>(configuration, script, parameterType);</span><br><span class="line"> <span class="keyword">return</span> builder.parseScriptNode();</span><br><span class="line"> }</span><br><span class="line"><span class="comment">// 初始化有一些逻辑</span></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">XMLScriptBuilder</span><span class="params">(Configuration configuration, XNode context, Class<?> parameterType)</span> {</span><br><span class="line"> <span class="built_in">super</span>(configuration);</span><br><span class="line"> <span class="built_in">this</span>.context = context;</span><br><span class="line"> <span class="built_in">this</span>.parameterType = parameterType;</span><br><span class="line"> <span class="comment">// 特别是这，我这次特意在 mapper 中加了 foreach，就是为了说下这一块的解析</span></span><br><span class="line"> initNodeHandlerMap();</span><br><span class="line"> }</span><br><span class="line"><span class="comment">// 设置各种类型的处理器</span></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">initNodeHandlerMap</span><span class="params">()</span> {</span><br><span class="line"> nodeHandlerMap.put(<span class="string">"trim"</span>, <span class="keyword">new</span> <span class="title class_">TrimHandler</span>());</span><br><span class="line"> nodeHandlerMap.put(<span class="string">"where"</span>, <span class="keyword">new</span> <span class="title class_">WhereHandler</span>());</span><br><span class="line"> nodeHandlerMap.put(<span class="string">"set"</span>, <span class="keyword">new</span> <span class="title class_">SetHandler</span>());</span><br><span class="line"> nodeHandlerMap.put(<span class="string">"foreach"</span>, <span class="keyword">new</span> <span class="title class_">ForEachHandler</span>());</span><br><span class="line"> nodeHandlerMap.put(<span class="string">"if"</span>, <span class="keyword">new</span> <span class="title class_">IfHandler</span>());</span><br><span class="line"> nodeHandlerMap.put(<span class="string">"choose"</span>, <span class="keyword">new</span> <span class="title class_">ChooseHandler</span>());</span><br><span class="line"> nodeHandlerMap.put(<span class="string">"when"</span>, <span class="keyword">new</span> <span class="title class_">IfHandler</span>());</span><br><span class="line"> nodeHandlerMap.put(<span class="string">"otherwise"</span>, <span class="keyword">new</span> <span class="title class_">OtherwiseHandler</span>());</span><br><span class="line"> nodeHandlerMap.put(<span class="string">"bind"</span>, <span class="keyword">new</span> <span class="title class_">BindHandler</span>());</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>初始化解析器以后就开始解析了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> SqlSource <span class="title function_">parseScriptNode</span><span class="params">()</span> {</span><br><span class="line"> <span class="comment">// 先是解析 parseDynamicTags</span></span><br><span class="line"> <span class="type">MixedSqlNode</span> <span class="variable">rootSqlNode</span> <span class="operator">=</span> parseDynamicTags(context);</span><br><span class="line"> SqlSource sqlSource;</span><br><span class="line"> <span class="keyword">if</span> (isDynamic) {</span><br><span class="line"> sqlSource = <span class="keyword">new</span> <span class="title class_">DynamicSqlSource</span>(configuration, rootSqlNode);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> sqlSource = <span class="keyword">new</span> <span class="title class_">RawSqlSource</span>(configuration, rootSqlNode, parameterType);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> sqlSource;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>但是这里可能做的事情比较多</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> MixedSqlNode <span class="title function_">parseDynamicTags</span><span class="params">(XNode node)</span> {</span><br><span class="line"> List<SqlNode> contents = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"> <span class="comment">// 获取子节点，这里可以把我 xml 中的 SELECT 语句分成三部分，第一部分是 select 到 in，然后是 foreach 部分，最后是\n结束符</span></span><br><span class="line"> <span class="type">NodeList</span> <span class="variable">children</span> <span class="operator">=</span> node.getNode().getChildNodes();</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < children.getLength(); i++) {</span><br><span class="line"> <span class="type">XNode</span> <span class="variable">child</span> <span class="operator">=</span> node.newXNode(children.item(i));</span><br><span class="line"> <span class="comment">// 第一个节点是个纯 text 节点就会走到这</span></span><br><span class="line"> <span class="keyword">if</span> (child.getNode().getNodeType() == Node.CDATA_SECTION_NODE || child.getNode().getNodeType() == Node.TEXT_NODE) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">data</span> <span class="operator">=</span> child.getStringBody(<span class="string">""</span>);</span><br><span class="line"> <span class="type">TextSqlNode</span> <span class="variable">textSqlNode</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">TextSqlNode</span>(data);</span><br><span class="line"> <span class="keyword">if</span> (textSqlNode.isDynamic()) {</span><br><span class="line"> contents.add(textSqlNode);</span><br><span class="line"> isDynamic = <span class="literal">true</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// 在 content 中添加这个 node</span></span><br><span class="line"> contents.add(<span class="keyword">new</span> <span class="title class_">StaticTextSqlNode</span>(data));</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (child.getNode().getNodeType() == Node.ELEMENT_NODE) { <span class="comment">// issue #628</span></span><br><span class="line"> <span class="comment">// 第二个节点是个带 foreach 的，是个内部元素节点</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">nodeName</span> <span class="operator">=</span> child.getNode().getNodeName();</span><br><span class="line"> <span class="comment">// 通过 nodeName 获取处理器</span></span><br><span class="line"> <span class="type">NodeHandler</span> <span class="variable">handler</span> <span class="operator">=</span> nodeHandlerMap.get(nodeName);</span><br><span class="line"> <span class="keyword">if</span> (handler == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">BuilderException</span>(<span class="string">"Unknown element <"</span> + nodeName + <span class="string">"> in SQL statement."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 调用处理器来处理</span></span><br><span class="line"> handler.handleNode(child, contents);</span><br><span class="line"> isDynamic = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 然后返回这个混合 sql 节点</span></span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">MixedSqlNode</span>(contents);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>再看下 handleNode 的逻辑</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">handleNode</span><span class="params">(XNode nodeToHandle, List<SqlNode> targetContents)</span> {</span><br><span class="line"> <span class="comment">// 又会套娃执行这里的 parseDynamicTags</span></span><br><span class="line"> <span class="type">MixedSqlNode</span> <span class="variable">mixedSqlNode</span> <span class="operator">=</span> parseDynamicTags(nodeToHandle);</span><br><span class="line"> <span class="type">String</span> <span class="variable">collection</span> <span class="operator">=</span> nodeToHandle.getStringAttribute(<span class="string">"collection"</span>);</span><br><span class="line"> <span class="type">Boolean</span> <span class="variable">nullable</span> <span class="operator">=</span> nodeToHandle.getBooleanAttribute(<span class="string">"nullable"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">item</span> <span class="operator">=</span> nodeToHandle.getStringAttribute(<span class="string">"item"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">index</span> <span class="operator">=</span> nodeToHandle.getStringAttribute(<span class="string">"index"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">open</span> <span class="operator">=</span> nodeToHandle.getStringAttribute(<span class="string">"open"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">close</span> <span class="operator">=</span> nodeToHandle.getStringAttribute(<span class="string">"close"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">separator</span> <span class="operator">=</span> nodeToHandle.getStringAttribute(<span class="string">"separator"</span>);</span><br><span class="line"> <span class="type">ForEachSqlNode</span> <span class="variable">forEachSqlNode</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ForEachSqlNode</span>(configuration, mixedSqlNode, collection, nullable, index, item, open, close, separator);</span><br><span class="line"> targetContents.add(forEachSqlNode);</span><br><span class="line"> }</span><br><span class="line"><span class="comment">// 这里走的逻辑不一样了</span></span><br><span class="line"><span class="keyword">protected</span> MixedSqlNode <span class="title function_">parseDynamicTags</span><span class="params">(XNode node)</span> {</span><br><span class="line"> List<SqlNode> contents = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"> <span class="comment">// 这里是 foreach 内部的，所以是个 text_node</span></span><br><span class="line"> <span class="type">NodeList</span> <span class="variable">children</span> <span class="operator">=</span> node.getNode().getChildNodes();</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < children.getLength(); i++) {</span><br><span class="line"> <span class="type">XNode</span> <span class="variable">child</span> <span class="operator">=</span> node.newXNode(children.item(i));</span><br><span class="line"> <span class="comment">// 第一个节点是个纯 text 节点就会走到这</span></span><br><span class="line"> <span class="keyword">if</span> (child.getNode().getNodeType() == Node.CDATA_SECTION_NODE || child.getNode().getNodeType() == Node.TEXT_NODE) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">data</span> <span class="operator">=</span> child.getStringBody(<span class="string">""</span>);</span><br><span class="line"> <span class="type">TextSqlNode</span> <span class="variable">textSqlNode</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">TextSqlNode</span>(data);</span><br><span class="line"> <span class="comment">// 判断是否动态是根据代码里是否有 ${}</span></span><br><span class="line"> <span class="keyword">if</span> (textSqlNode.isDynamic()) {</span><br><span class="line"> contents.add(textSqlNode);</span><br><span class="line"> isDynamic = <span class="literal">true</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// 所以还是会走到这</span></span><br><span class="line"> <span class="comment">// 在 content 中添加这个 node</span></span><br><span class="line"> contents.add(<span class="keyword">new</span> <span class="title class_">StaticTextSqlNode</span>(data));</span><br><span class="line"> }</span><br><span class="line"><span class="comment">// 最后继续包装成 MixedSqlNode</span></span><br><span class="line"><span class="comment">// 再回到这里</span></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">handleNode</span><span class="params">(XNode nodeToHandle, List<SqlNode> targetContents)</span> {</span><br><span class="line"> <span class="type">MixedSqlNode</span> <span class="variable">mixedSqlNode</span> <span class="operator">=</span> parseDynamicTags(nodeToHandle);</span><br><span class="line"> <span class="comment">// 处理 foreach 内部的各个变量</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">collection</span> <span class="operator">=</span> nodeToHandle.getStringAttribute(<span class="string">"collection"</span>);</span><br><span class="line"> <span class="type">Boolean</span> <span class="variable">nullable</span> <span class="operator">=</span> nodeToHandle.getBooleanAttribute(<span class="string">"nullable"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">item</span> <span class="operator">=</span> nodeToHandle.getStringAttribute(<span class="string">"item"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">index</span> <span class="operator">=</span> nodeToHandle.getStringAttribute(<span class="string">"index"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">open</span> <span class="operator">=</span> nodeToHandle.getStringAttribute(<span class="string">"open"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">close</span> <span class="operator">=</span> nodeToHandle.getStringAttribute(<span class="string">"close"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">separator</span> <span class="operator">=</span> nodeToHandle.getStringAttribute(<span class="string">"separator"</span>);</span><br><span class="line"> <span class="type">ForEachSqlNode</span> <span class="variable">forEachSqlNode</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ForEachSqlNode</span>(configuration, mixedSqlNode, collection, nullable, index, item, open, close, separator);</span><br><span class="line"> targetContents.add(forEachSqlNode);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>再回过来</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> SqlSource <span class="title function_">parseScriptNode</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">MixedSqlNode</span> <span class="variable">rootSqlNode</span> <span class="operator">=</span> parseDynamicTags(context);</span><br><span class="line"> SqlSource sqlSource;</span><br><span class="line"> <span class="comment">// 因为在 foreach 节点处理时直接是把 isDynamic 置成了 true</span></span><br><span class="line"> <span class="keyword">if</span> (isDynamic) {</span><br><span class="line"> <span class="comment">// 所以是个 DynamicSqlSource</span></span><br><span class="line"> sqlSource = <span class="keyword">new</span> <span class="title class_">DynamicSqlSource</span>(configuration, rootSqlNode);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> sqlSource = <span class="keyword">new</span> <span class="title class_">RawSqlSource</span>(configuration, rootSqlNode, parameterType);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> sqlSource;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里就做完了预处理工作，真正在执行的执行的时候还需要进一步解析</p>
|
|
<p>因为前面讲过很多了，所以直接跳到这里</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <T> T <span class="title function_">selectOne</span><span class="params">(String statement, Object parameter)</span> {</span><br><span class="line"> <span class="comment">// Popular vote was to return null on 0 results and throw exception on too many.</span></span><br><span class="line"> <span class="comment">// 都知道是在这进去</span></span><br><span class="line"> List<T> list = <span class="built_in">this</span>.selectList(statement, parameter);</span><br><span class="line"> <span class="keyword">if</span> (list.size() == <span class="number">1</span>) {</span><br><span class="line"> <span class="keyword">return</span> list.get(<span class="number">0</span>);</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (list.size() > <span class="number">1</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">TooManyResultsException</span>(<span class="string">"Expected one result (or null) to be returned by selectOne(), but found: "</span> + list.size());</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <E> List<E> <span class="title function_">selectList</span><span class="params">(String statement, Object parameter)</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.selectList(statement, parameter, RowBounds.DEFAULT);</span><br><span class="line"> }</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <E> List<E> <span class="title function_">selectList</span><span class="params">(String statement, Object parameter, RowBounds rowBounds)</span> {</span><br><span class="line"> <span class="keyword">return</span> selectList(statement, parameter, rowBounds, Executor.NO_RESULT_HANDLER);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">private</span> <E> List<E> <span class="title function_">selectList</span><span class="params">(String statement, Object parameter, RowBounds rowBounds, ResultHandler handler)</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 前面也讲过这个，</span></span><br><span class="line"> <span class="type">MappedStatement</span> <span class="variable">ms</span> <span class="operator">=</span> configuration.getMappedStatement(statement);</span><br><span class="line"> <span class="keyword">return</span> executor.query(ms, wrapCollection(parameter), rowBounds, handler);</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> ExceptionFactory.wrapException(<span class="string">"Error querying database. Cause: "</span> + e, e);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> ErrorContext.instance().reset();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 包括这里，是调用的org.apache.ibatis.executor.CachingExecutor#query(org.apache.ibatis.mapping.MappedStatement, java.lang.Object, org.apache.ibatis.session.RowBounds, org.apache.ibatis.session.ResultHandler)</span></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <E> List<E> <span class="title function_">query</span><span class="params">(MappedStatement ms, Object parameterObject, RowBounds rowBounds, ResultHandler resultHandler)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="type">BoundSql</span> <span class="variable">boundSql</span> <span class="operator">=</span> ms.getBoundSql(parameterObject);</span><br><span class="line"> <span class="type">CacheKey</span> <span class="variable">key</span> <span class="operator">=</span> createCacheKey(ms, parameterObject, rowBounds, boundSql);</span><br><span class="line"> <span class="keyword">return</span> query(ms, parameterObject, rowBounds, resultHandler, key, boundSql);</span><br><span class="line"> }</span><br><span class="line"><span class="comment">// 然后是获取 BoundSql</span></span><br><span class="line"> <span class="keyword">public</span> BoundSql <span class="title function_">getBoundSql</span><span class="params">(Object parameterObject)</span> {</span><br><span class="line"> <span class="type">BoundSql</span> <span class="variable">boundSql</span> <span class="operator">=</span> sqlSource.getBoundSql(parameterObject);</span><br><span class="line"> List<ParameterMapping> parameterMappings = boundSql.getParameterMappings();</span><br><span class="line"> <span class="keyword">if</span> (parameterMappings == <span class="literal">null</span> || parameterMappings.isEmpty()) {</span><br><span class="line"> boundSql = <span class="keyword">new</span> <span class="title class_">BoundSql</span>(configuration, boundSql.getSql(), parameterMap.getParameterMappings(), parameterObject);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// check for nested result maps in parameter mappings (issue #30)</span></span><br><span class="line"> <span class="keyword">for</span> (ParameterMapping pm : boundSql.getParameterMappings()) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">rmId</span> <span class="operator">=</span> pm.getResultMapId();</span><br><span class="line"> <span class="keyword">if</span> (rmId != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">ResultMap</span> <span class="variable">rm</span> <span class="operator">=</span> configuration.getResultMap(rmId);</span><br><span class="line"> <span class="keyword">if</span> (rm != <span class="literal">null</span>) {</span><br><span class="line"> hasNestedResultMaps |= rm.hasNestedResultMaps();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> boundSql;</span><br><span class="line"> }</span><br><span class="line"><span class="comment">// 因为前面讲了是生成的 DynamicSqlSource，所以也是调用这个的 getBoundSql</span></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> BoundSql <span class="title function_">getBoundSql</span><span class="params">(Object parameterObject)</span> {</span><br><span class="line"> <span class="type">DynamicContext</span> <span class="variable">context</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">DynamicContext</span>(configuration, parameterObject);</span><br><span class="line"> <span class="comment">// 重点关注着</span></span><br><span class="line"> rootSqlNode.apply(context);</span><br><span class="line"> <span class="type">SqlSourceBuilder</span> <span class="variable">sqlSourceParser</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SqlSourceBuilder</span>(configuration);</span><br><span class="line"> Class<?> parameterType = parameterObject == <span class="literal">null</span> ? Object.class : parameterObject.getClass();</span><br><span class="line"> <span class="type">SqlSource</span> <span class="variable">sqlSource</span> <span class="operator">=</span> sqlSourceParser.parse(context.getSql(), parameterType, context.getBindings());</span><br><span class="line"> <span class="type">BoundSql</span> <span class="variable">boundSql</span> <span class="operator">=</span> sqlSource.getBoundSql(parameterObject);</span><br><span class="line"> context.getBindings().forEach(boundSql::setAdditionalParameter);</span><br><span class="line"> <span class="keyword">return</span> boundSql;</span><br><span class="line"> }</span><br><span class="line"><span class="comment">// 继续是这个 DynamicSqlNode 的 apply</span></span><br><span class="line"> <span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">apply</span><span class="params">(DynamicContext context)</span> {</span><br><span class="line"> contents.forEach(node -> node.apply(context));</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"><span class="comment">// 看下面的图</span></span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/blog/BFlyxu.png" alt="image"></p>
|
|
<p>我们重点看 foreach 的逻辑</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">apply</span><span class="params">(DynamicContext context)</span> {</span><br><span class="line"> Map<String, Object> bindings = context.getBindings();</span><br><span class="line"> <span class="keyword">final</span> Iterable<?> iterable = evaluator.evaluateIterable(collectionExpression, bindings,</span><br><span class="line"> Optional.ofNullable(nullable).orElseGet(configuration::isNullableOnForEach));</span><br><span class="line"> <span class="keyword">if</span> (iterable == <span class="literal">null</span> || !iterable.iterator().hasNext()) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">first</span> <span class="operator">=</span> <span class="literal">true</span>;</span><br><span class="line"> <span class="comment">// 开始符号</span></span><br><span class="line"> applyOpen(context);</span><br><span class="line"> <span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">for</span> (Object o : iterable) {</span><br><span class="line"> <span class="type">DynamicContext</span> <span class="variable">oldContext</span> <span class="operator">=</span> context;</span><br><span class="line"> <span class="keyword">if</span> (first || separator == <span class="literal">null</span>) {</span><br><span class="line"> context = <span class="keyword">new</span> <span class="title class_">PrefixedContext</span>(context, <span class="string">""</span>);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> context = <span class="keyword">new</span> <span class="title class_">PrefixedContext</span>(context, separator);</span><br><span class="line"> }</span><br><span class="line"> <span class="type">int</span> <span class="variable">uniqueNumber</span> <span class="operator">=</span> context.getUniqueNumber();</span><br><span class="line"> <span class="comment">// Issue #709</span></span><br><span class="line"> <span class="keyword">if</span> (o <span class="keyword">instanceof</span> Map.Entry) {</span><br><span class="line"> <span class="meta">@SuppressWarnings("unchecked")</span></span><br><span class="line"> Map.Entry<Object, Object> mapEntry = (Map.Entry<Object, Object>) o;</span><br><span class="line"> applyIndex(context, mapEntry.getKey(), uniqueNumber);</span><br><span class="line"> applyItem(context, mapEntry.getValue(), uniqueNumber);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> applyIndex(context, i, uniqueNumber);</span><br><span class="line"> applyItem(context, o, uniqueNumber);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 转换变量名，变成这种形式 select * from student where id in</span></span><br><span class="line"> <span class="comment">// ( </span></span><br><span class="line"> <span class="comment">// #{__frch_id_0}</span></span><br><span class="line"> <span class="comment">// )</span></span><br><span class="line"> contents.apply(<span class="keyword">new</span> <span class="title class_">FilteredDynamicContext</span>(configuration, context, index, item, uniqueNumber));</span><br><span class="line"> <span class="keyword">if</span> (first) {</span><br><span class="line"> first = !((PrefixedContext) context).isPrefixApplied();</span><br><span class="line"> }</span><br><span class="line"> context = oldContext;</span><br><span class="line"> i++;</span><br><span class="line"> }</span><br><span class="line"> applyClose(context);</span><br><span class="line"> context.getBindings().remove(item);</span><br><span class="line"> context.getBindings().remove(index);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"><span class="comment">// 回到外层就会调用 parse 方法， 把#{} 这段替换成 ?</span></span><br><span class="line"><span class="keyword">public</span> SqlSource <span class="title function_">parse</span><span class="params">(String originalSql, Class<?> parameterType, Map<String, Object> additionalParameters)</span> {</span><br><span class="line"> <span class="type">ParameterMappingTokenHandler</span> <span class="variable">handler</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ParameterMappingTokenHandler</span>(configuration, parameterType, additionalParameters);</span><br><span class="line"> <span class="type">GenericTokenParser</span> <span class="variable">parser</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">GenericTokenParser</span>(<span class="string">"#{"</span>, <span class="string">"}"</span>, handler);</span><br><span class="line"> String sql;</span><br><span class="line"> <span class="keyword">if</span> (configuration.isShrinkWhitespacesInSql()) {</span><br><span class="line"> sql = parser.parse(removeExtraWhitespaces(originalSql));</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> sql = parser.parse(originalSql);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">StaticSqlSource</span>(configuration, sql, handler.getParameterMappings());</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/blog/nE19Uv.png" alt="image"></p>
|
|
<p>可以看到这里，然后再进行替换</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/la6tMs.png" alt="image"></p>
|
|
<p>真实的从 <code>?</code> 替换成具体的变量值，是在这里<br><code>org.apache.ibatis.executor.SimpleExecutor#doQuery</code><br>调用了 </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> Statement <span class="title function_">prepareStatement</span><span class="params">(StatementHandler handler, Log statementLog)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> Statement stmt;</span><br><span class="line"> <span class="type">Connection</span> <span class="variable">connection</span> <span class="operator">=</span> getConnection(statementLog);</span><br><span class="line"> stmt = handler.prepare(connection, transaction.getTimeout());</span><br><span class="line"> handler.parameterize(stmt);</span><br><span class="line"> <span class="keyword">return</span> stmt;</span><br><span class="line"> }</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">parameterize</span><span class="params">(Statement statement)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> parameterHandler.setParameters((PreparedStatement) statement);</span><br><span class="line"> }</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">setParameters</span><span class="params">(PreparedStatement ps)</span> {</span><br><span class="line"> ErrorContext.instance().activity(<span class="string">"setting parameters"</span>).object(mappedStatement.getParameterMap().getId());</span><br><span class="line"> List<ParameterMapping> parameterMappings = boundSql.getParameterMappings();</span><br><span class="line"> <span class="keyword">if</span> (parameterMappings != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < parameterMappings.size(); i++) {</span><br><span class="line"> <span class="type">ParameterMapping</span> <span class="variable">parameterMapping</span> <span class="operator">=</span> parameterMappings.get(i);</span><br><span class="line"> <span class="keyword">if</span> (parameterMapping.getMode() != ParameterMode.OUT) {</span><br><span class="line"> Object value;</span><br><span class="line"> <span class="type">String</span> <span class="variable">propertyName</span> <span class="operator">=</span> parameterMapping.getProperty();</span><br><span class="line"> <span class="keyword">if</span> (boundSql.hasAdditionalParameter(propertyName)) { <span class="comment">// issue #448 ask first for additional params</span></span><br><span class="line"> value = boundSql.getAdditionalParameter(propertyName);</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (parameterObject == <span class="literal">null</span>) {</span><br><span class="line"> value = <span class="literal">null</span>;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (typeHandlerRegistry.hasTypeHandler(parameterObject.getClass())) {</span><br><span class="line"> value = parameterObject;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="type">MetaObject</span> <span class="variable">metaObject</span> <span class="operator">=</span> configuration.newMetaObject(parameterObject);</span><br><span class="line"> value = metaObject.getValue(propertyName);</span><br><span class="line"> }</span><br><span class="line"> <span class="type">TypeHandler</span> <span class="variable">typeHandler</span> <span class="operator">=</span> parameterMapping.getTypeHandler();</span><br><span class="line"> <span class="type">JdbcType</span> <span class="variable">jdbcType</span> <span class="operator">=</span> parameterMapping.getJdbcType();</span><br><span class="line"> <span class="keyword">if</span> (value == <span class="literal">null</span> && jdbcType == <span class="literal">null</span>) {</span><br><span class="line"> jdbcType = configuration.getJdbcTypeForNull();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// --------------------------> </span></span><br><span class="line"> <span class="comment">// 替换变量</span></span><br><span class="line"> typeHandler.setParameter(ps, i + <span class="number">1</span>, value, jdbcType);</span><br><span class="line"> } <span class="keyword">catch</span> (TypeException | SQLException e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">TypeException</span>(<span class="string">"Could not set parameters for mapping: "</span> + parameterMapping + <span class="string">". Cause: "</span> + e, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Mybatis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Mysql</tag>
|
|
<tag>Mybatis</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>mybatis系列-sql 类的简单使用</title>
|
|
<url>/2023/03/12/mybatis%E7%B3%BB%E5%88%97-sql-%E7%B1%BB%E7%9A%84%E7%AE%80%E5%8D%95%E4%BD%BF%E7%94%A8/</url>
|
|
<content><![CDATA[<p>mybatis 还有个比较有趣的功能，就是使用 SQL 类生成 sql，有点类似于 hibernate 或者像 php 的 laravel 框架等的，就是把sql 这种放在 xml 里或者代码里直接写 sql 用对象的形式</p>
|
|
<h3 id="select语句"><a href="#select语句" class="headerlink" title="select语句"></a>select语句</h3><p>比如这样</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">selectSql</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SQL</span>() {{</span><br><span class="line"> SELECT(<span class="string">"id"</span>, <span class="string">"name"</span>);</span><br><span class="line"> FROM(<span class="string">"student"</span>);</span><br><span class="line"> WHERE(<span class="string">"id = #{id}"</span>);</span><br><span class="line"> }}.toString();</span><br><span class="line"> System.out.println(selectSql);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>打印出来就是</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">SELECT id, name</span><br><span class="line">FROM student</span><br><span class="line">WHERE (id = #{id})</span><br></pre></td></tr></table></figure>
|
|
<p>应付简单的 sql 查询基本都可以这么解决，如果习惯这种模式，还是不错的，<br>其实以面向对象的编程模式来说，这样是比较符合面向对象的，先不深入的解析这块的源码，先从使用角度讲一下</p>
|
|
<h3 id="比如-update-语句"><a href="#比如-update-语句" class="headerlink" title="比如 update 语句"></a>比如 update 语句</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">String</span> <span class="variable">updateSql</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SQL</span>() {{</span><br><span class="line"> UPDATE(<span class="string">"student"</span>);</span><br><span class="line"> SET(<span class="string">"name = #{name}"</span>);</span><br><span class="line"> WHERE(<span class="string">"id = #{id}"</span>);</span><br><span class="line"> }}.toString();</span><br></pre></td></tr></table></figure>
|
|
<p>打印输出就是</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">UPDATE student</span><br><span class="line"><span class="type">SET</span> <span class="variable">name</span> <span class="operator">=</span> #{name}</span><br><span class="line">WHERE (id = #{id})</span><br></pre></td></tr></table></figure>
|
|
|
|
<h3 id="insert-语句"><a href="#insert-语句" class="headerlink" title="insert 语句"></a>insert 语句</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">String</span> <span class="variable">insertSql</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SQL</span>() {{</span><br><span class="line"> INSERT_INTO(<span class="string">"student"</span>);</span><br><span class="line"> VALUES(<span class="string">"name"</span>, <span class="string">"#{name}"</span>);</span><br><span class="line"> VALUES(<span class="string">"age"</span>, <span class="string">"#{age}"</span>);</span><br><span class="line"> }}.toString();</span><br><span class="line"> System.out.println(insertSql);</span><br></pre></td></tr></table></figure>
|
|
<p>打印输出</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">INSERT INTO <span class="title function_">student</span></span><br><span class="line"> <span class="params">(name, age)</span></span><br><span class="line">VALUES (#{name}, #{age})</span><br></pre></td></tr></table></figure>
|
|
<h3 id="delete语句"><a href="#delete语句" class="headerlink" title="delete语句"></a>delete语句</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">String</span> <span class="variable">deleteSql</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SQL</span>() {{</span><br><span class="line"> DELETE_FROM(<span class="string">"student"</span>);</span><br><span class="line"> WHERE(<span class="string">"id = #{id}"</span>);</span><br><span class="line"> }}.toString();</span><br><span class="line"> System.out.println(deleteSql);</span><br></pre></td></tr></table></figure>
|
|
<p>打印输出</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">DELETE FROM student</span><br><span class="line"><span class="title function_">WHERE</span> <span class="params">(id = #{id})</span></span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Mybatis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Mysql</tag>
|
|
<tag>Mybatis</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>mybatis系列-sql 类的简要分析</title>
|
|
<url>/2023/03/19/mybatis%E7%B3%BB%E5%88%97-sql-%E7%B1%BB%E7%9A%84%E7%AE%80%E8%A6%81%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<p>上次就比较简单的讲了使用，这块也比较简单，因为封装得不是很复杂，首先我们从 select 作为入口来看看，这个具体的实现，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">String</span> <span class="variable">selectSql</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SQL</span>() {{</span><br><span class="line"> SELECT(<span class="string">"id"</span>, <span class="string">"name"</span>);</span><br><span class="line"> FROM(<span class="string">"student"</span>);</span><br><span class="line"> WHERE(<span class="string">"id = #{id}"</span>);</span><br><span class="line"> }}.toString();</span><br></pre></td></tr></table></figure>
|
|
<p><code>SELECT</code> 方法的实现，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> T <span class="title function_">SELECT</span><span class="params">(String... columns)</span> {</span><br><span class="line"> sql().statementType = SQLStatement.StatementType.SELECT;</span><br><span class="line"> sql().select.addAll(Arrays.asList(columns));</span><br><span class="line"> <span class="keyword">return</span> getSelf();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>statementType是个枚举</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">enum</span> <span class="title class_">StatementType</span> {</span><br><span class="line"> DELETE, INSERT, SELECT, UPDATE</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>那这个就是个 select 语句，然后会把参数转成 list 添加到 <code>select</code> 变量里，<br>然后是 from 语句，这个大概也能猜到就是设置下表名，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> T <span class="title function_">FROM</span><span class="params">(String table)</span> {</span><br><span class="line"> sql().tables.add(table);</span><br><span class="line"> <span class="keyword">return</span> getSelf();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>往 tables 里添加了 table，这个 tables 是什么呢<br>这里也可以看下所有的变量，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">StatementType statementType;</span><br><span class="line">List<String> sets = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line">List<String> select = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line">List<String> tables = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line">List<String> join = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line">List<String> innerJoin = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line">List<String> outerJoin = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line">List<String> leftOuterJoin = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line">List<String> rightOuterJoin = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line">List<String> where = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line">List<String> having = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line">List<String> groupBy = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line">List<String> orderBy = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line">List<String> lastList = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line">List<String> columns = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line">List<List<String>> valuesList = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br></pre></td></tr></table></figure>
|
|
<p>可以看到是一堆 List 先暂存这些sql 片段，然后再拼装成 sql 语句，<br>因为它重写了 toString 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> String <span class="title function_">toString</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">StringBuilder</span> <span class="variable">sb</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">StringBuilder</span>();</span><br><span class="line"> sql().sql(sb);</span><br><span class="line"> <span class="keyword">return</span> sb.toString();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>调用的 sql 方法是</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> String <span class="title function_">sql</span><span class="params">(Appendable a)</span> {</span><br><span class="line"> <span class="type">SafeAppendable</span> <span class="variable">builder</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SafeAppendable</span>(a);</span><br><span class="line"> <span class="keyword">if</span> (statementType == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> String answer;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">switch</span> (statementType) {</span><br><span class="line"> <span class="keyword">case</span> DELETE:</span><br><span class="line"> answer = deleteSQL(builder);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">case</span> INSERT:</span><br><span class="line"> answer = insertSQL(builder);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">case</span> SELECT:</span><br><span class="line"> answer = selectSQL(builder);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">case</span> UPDATE:</span><br><span class="line"> answer = updateSQL(builder);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">default</span>:</span><br><span class="line"> answer = <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> answer;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>根据上面的 statementType判断是个什么 sql，我们这个是 selectSQL 就走的 SELECT 这个分支</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> String <span class="title function_">selectSQL</span><span class="params">(SafeAppendable builder)</span> {</span><br><span class="line"> <span class="keyword">if</span> (distinct) {</span><br><span class="line"> sqlClause(builder, <span class="string">"SELECT DISTINCT"</span>, select, <span class="string">""</span>, <span class="string">""</span>, <span class="string">", "</span>);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> sqlClause(builder, <span class="string">"SELECT"</span>, select, <span class="string">""</span>, <span class="string">""</span>, <span class="string">", "</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> sqlClause(builder, <span class="string">"FROM"</span>, tables, <span class="string">""</span>, <span class="string">""</span>, <span class="string">", "</span>);</span><br><span class="line"> joins(builder);</span><br><span class="line"> sqlClause(builder, <span class="string">"WHERE"</span>, where, <span class="string">"("</span>, <span class="string">")"</span>, <span class="string">" AND "</span>);</span><br><span class="line"> sqlClause(builder, <span class="string">"GROUP BY"</span>, groupBy, <span class="string">""</span>, <span class="string">""</span>, <span class="string">", "</span>);</span><br><span class="line"> sqlClause(builder, <span class="string">"HAVING"</span>, having, <span class="string">"("</span>, <span class="string">")"</span>, <span class="string">" AND "</span>);</span><br><span class="line"> sqlClause(builder, <span class="string">"ORDER BY"</span>, orderBy, <span class="string">""</span>, <span class="string">""</span>, <span class="string">", "</span>);</span><br><span class="line"> limitingRowsStrategy.appendClause(builder, offset, limit);</span><br><span class="line"> <span class="keyword">return</span> builder.toString();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>上面的可以看出来就是按我们常规的 sql 理解顺序来处理<br>就是<code>select ... from ... where ... </code>这样子<br>再看下 sqlClause 的代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">sqlClause</span><span class="params">(SafeAppendable builder, String keyword, List<String> parts, String open, String close,</span></span><br><span class="line"><span class="params"> String conjunction)</span> {</span><br><span class="line"> <span class="keyword">if</span> (!parts.isEmpty()) {</span><br><span class="line"> <span class="keyword">if</span> (!builder.isEmpty()) {</span><br><span class="line"> builder.append(<span class="string">"\n"</span>);</span><br><span class="line"> }</span><br><span class="line"> builder.append(keyword);</span><br><span class="line"> builder.append(<span class="string">" "</span>);</span><br><span class="line"> builder.append(open);</span><br><span class="line"> <span class="type">String</span> <span class="variable">last</span> <span class="operator">=</span> <span class="string">"________"</span>;</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>, n = parts.size(); i < n; i++) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">part</span> <span class="operator">=</span> parts.get(i);</span><br><span class="line"> <span class="keyword">if</span> (i > <span class="number">0</span> && !part.equals(AND) && !part.equals(OR) && !last.equals(AND) && !last.equals(OR)) {</span><br><span class="line"> builder.append(conjunction);</span><br><span class="line"> }</span><br><span class="line"> builder.append(part);</span><br><span class="line"> last = part;</span><br><span class="line"> }</span><br><span class="line"> builder.append(close);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里的拼接方式还需要判断 AND 和 OR 的判断逻辑，其他就没什么特别的了，只是where 语句中的 lastList 不知道是干嘛的，好像只有添加跟赋值的操作，有知道的大神也可以评论指导下</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Mybatis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Mysql</tag>
|
|
<tag>Mybatis</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>mybatis系列-typeAliases系统</title>
|
|
<url>/2023/01/01/mybatis%E7%B3%BB%E5%88%97-typeAliases%E7%B3%BB%E7%BB%9F/</url>
|
|
<content><![CDATA[<p>其实前面已经聊到过这个概念，在mybatis的配置中，以及一些初始化逻辑都是用了typeAliases，</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">typeAliases</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">typeAlias</span> <span class="attr">alias</span>=<span class="string">"Author"</span> <span class="attr">type</span>=<span class="string">"domain.blog.Author"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">typeAlias</span> <span class="attr">alias</span>=<span class="string">"Blog"</span> <span class="attr">type</span>=<span class="string">"domain.blog.Blog"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">typeAlias</span> <span class="attr">alias</span>=<span class="string">"Comment"</span> <span class="attr">type</span>=<span class="string">"domain.blog.Comment"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">typeAlias</span> <span class="attr">alias</span>=<span class="string">"Post"</span> <span class="attr">type</span>=<span class="string">"domain.blog.Post"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">typeAlias</span> <span class="attr">alias</span>=<span class="string">"Section"</span> <span class="attr">type</span>=<span class="string">"domain.blog.Section"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">typeAlias</span> <span class="attr">alias</span>=<span class="string">"Tag"</span> <span class="attr">type</span>=<span class="string">"domain.blog.Tag"</span>/></span></span><br><span class="line"><span class="tag"></<span class="name">typeAliases</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>可以在这里注册类型别名，然后在mybatis中配置使用时，可以简化这些类型的使用，其底层逻辑主要是一个map，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">TypeAliasRegistry</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> Map<String, Class<?>> typeAliases = <span class="keyword">new</span> <span class="title class_">HashMap</span><>();</span><br></pre></td></tr></table></figure>
|
|
<p>以string作为key，class对象作为value，比如我们在一开始使用的配置文件</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">dataSource</span> <span class="attr">type</span>=<span class="string">"POOLED"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"driver"</span> <span class="attr">value</span>=<span class="string">"${driver}"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"url"</span> <span class="attr">value</span>=<span class="string">"${url}"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"username"</span> <span class="attr">value</span>=<span class="string">"${username}"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"password"</span> <span class="attr">value</span>=<span class="string">"${password}"</span>/></span></span><br><span class="line"><span class="tag"></<span class="name">dataSource</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>这里使用的dataSource是POOLED，那它肯定是个别名或者需要对应处理<br>而这个别名就是在Configuration的构造方法里初始化</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">Configuration</span><span class="params">()</span> {</span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"JDBC"</span>, JdbcTransactionFactory.class);</span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"MANAGED"</span>, ManagedTransactionFactory.class);</span><br><span class="line"></span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"JNDI"</span>, JndiDataSourceFactory.class);</span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"POOLED"</span>, PooledDataSourceFactory.class);</span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"UNPOOLED"</span>, UnpooledDataSourceFactory.class);</span><br><span class="line"></span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"PERPETUAL"</span>, PerpetualCache.class);</span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"FIFO"</span>, FifoCache.class);</span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"LRU"</span>, LruCache.class);</span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"SOFT"</span>, SoftCache.class);</span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"WEAK"</span>, WeakCache.class);</span><br><span class="line"></span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"DB_VENDOR"</span>, VendorDatabaseIdProvider.class);</span><br><span class="line"></span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"XML"</span>, XMLLanguageDriver.class);</span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"RAW"</span>, RawLanguageDriver.class);</span><br><span class="line"></span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"SLF4J"</span>, Slf4jImpl.class);</span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"COMMONS_LOGGING"</span>, JakartaCommonsLoggingImpl.class);</span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"LOG4J"</span>, Log4jImpl.class);</span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"LOG4J2"</span>, Log4j2Impl.class);</span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"JDK_LOGGING"</span>, Jdk14LoggingImpl.class);</span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"STDOUT_LOGGING"</span>, StdOutImpl.class);</span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"NO_LOGGING"</span>, NoLoggingImpl.class);</span><br><span class="line"></span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"CGLIB"</span>, CglibProxyFactory.class);</span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"JAVASSIST"</span>, JavassistProxyFactory.class);</span><br><span class="line"></span><br><span class="line"> languageRegistry.setDefaultDriverClass(XMLLanguageDriver.class);</span><br><span class="line"> languageRegistry.register(RawLanguageDriver.class);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>正是通过<code>typeAliasRegistry.registerAlias("POOLED", PooledDataSourceFactory.class);</code>这一行，注册了<br><code>POOLED</code>对应的别名类型是<code>PooledDataSourceFactory.class</code><br>具体的注册方法是在</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">registerAlias</span><span class="params">(String alias, Class<?> value)</span> {</span><br><span class="line"> <span class="keyword">if</span> (alias == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">TypeException</span>(<span class="string">"The parameter alias cannot be null"</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// issue #748</span></span><br><span class="line"> <span class="comment">// 转换成小写，</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">key</span> <span class="operator">=</span> alias.toLowerCase(Locale.ENGLISH);</span><br><span class="line"> <span class="comment">// 判断是否已经注册过了</span></span><br><span class="line"> <span class="keyword">if</span> (typeAliases.containsKey(key) && typeAliases.get(key) != <span class="literal">null</span> && !typeAliases.get(key).equals(value)) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">TypeException</span>(<span class="string">"The alias '"</span> + alias + <span class="string">"' is already mapped to the value '"</span> + typeAliases.get(key).getName() + <span class="string">"'."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 放进map里</span></span><br><span class="line"> typeAliases.put(key, value);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>而获取的逻辑在这</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <T> Class<T> <span class="title function_">resolveAlias</span><span class="params">(String string)</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (string == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// issue #748</span></span><br><span class="line"> <span class="comment">// 同样的转成小写</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">key</span> <span class="operator">=</span> string.toLowerCase(Locale.ENGLISH);</span><br><span class="line"> Class<T> value;</span><br><span class="line"> <span class="keyword">if</span> (typeAliases.containsKey(key)) {</span><br><span class="line"> value = (Class<T>) typeAliases.get(key);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// 这里还有从路径下处理的逻辑</span></span><br><span class="line"> value = (Class<T>) Resources.classForName(string);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> value;</span><br><span class="line"> } <span class="keyword">catch</span> (ClassNotFoundException e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">TypeException</span>(<span class="string">"Could not resolve type alias '"</span> + string + <span class="string">"'. Cause: "</span> + e, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>逻辑比较简单，但是在mybatis中也是不可或缺的一块概念</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Mybatis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Mysql</tag>
|
|
<tag>Mybatis</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>mybatis系列-入门篇</title>
|
|
<url>/2022/11/27/mybatis%E7%B3%BB%E5%88%97-%E5%85%A5%E9%97%A8%E7%AF%87/</url>
|
|
<content><![CDATA[<p>mybatis是我们比较常用的orm框架，下面是官网的介绍</p>
|
|
<blockquote>
|
|
<p> MyBatis 是一款优秀的持久层框架，它支持自定义 SQL、存储过程以及高级映射。MyBatis 免除了几乎所有的 JDBC 代码以及设置参数和获取结果集的工作。MyBatis 可以通过简单的 XML 或注解来配置和映射原始类型、接口和 Java POJO（Plain Old Java Objects，普通老式 Java 对象）为数据库中的记录。 </p>
|
|
</blockquote>
|
|
<p>mybatis一大特点，或者说比较为人熟知的应该就是比 hibernate 是更轻量化，为国人所爱好的orm框架，对于hibernate目前还没有深入的拆解过，后续可以也写一下，在使用体验上觉得是个比较精巧的框架，看代码也比较容易，所以就想写个系列，第一篇先是介绍下使用<br>根据官网的文档上我们先来尝试一下简单使用<br>首先我们有个简单的配置，这个文件是<code>mybatis-config.xml</code></p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="meta"><?xml version=<span class="string">"1.0"</span> encoding=<span class="string">"UTF-8"</span> ?></span></span><br><span class="line"><span class="meta"><!DOCTYPE <span class="keyword">configuration</span></span></span><br><span class="line"><span class="meta"> <span class="keyword">PUBLIC</span> <span class="string">"-//mybatis.org//DTD Config 3.0//EN"</span></span></span><br><span class="line"><span class="meta"> <span class="string">"https://mybatis.org/dtd/mybatis-3-config.dtd"</span>></span></span><br><span class="line"><span class="tag"><<span class="name">configuration</span>></span></span><br><span class="line"> <span class="comment"><!-- 需要加入的properties--></span></span><br><span class="line"> <span class="tag"><<span class="name">properties</span> <span class="attr">resource</span>=<span class="string">"application-development.properties"</span>/></span></span><br><span class="line"> <span class="comment"><!-- 指出使用哪个环境，默认是development--></span></span><br><span class="line"> <span class="tag"><<span class="name">environments</span> <span class="attr">default</span>=<span class="string">"development"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">environment</span> <span class="attr">id</span>=<span class="string">"development"</span>></span></span><br><span class="line"> <span class="comment"><!-- 指定事务管理器类型--></span></span><br><span class="line"> <span class="tag"><<span class="name">transactionManager</span> <span class="attr">type</span>=<span class="string">"JDBC"</span>/></span></span><br><span class="line"> <span class="comment"><!-- 指定数据源类型--></span></span><br><span class="line"> <span class="tag"><<span class="name">dataSource</span> <span class="attr">type</span>=<span class="string">"POOLED"</span>></span></span><br><span class="line"> <span class="comment"><!-- 下面就是具体的参数占位了--></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"driver"</span> <span class="attr">value</span>=<span class="string">"${driver}"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"url"</span> <span class="attr">value</span>=<span class="string">"${url}"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"username"</span> <span class="attr">value</span>=<span class="string">"${username}"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"password"</span> <span class="attr">value</span>=<span class="string">"${password}"</span>/></span></span><br><span class="line"> <span class="tag"></<span class="name">dataSource</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">environment</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">environments</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">mappers</span>></span></span><br><span class="line"> <span class="comment"><!-- 指定mapper xml的位置或文件--></span></span><br><span class="line"> <span class="tag"><<span class="name">mapper</span> <span class="attr">resource</span>=<span class="string">"mapper/StudentMapper.xml"</span>/></span></span><br><span class="line"> <span class="tag"></<span class="name">mappers</span>></span></span><br><span class="line"><span class="tag"></<span class="name">configuration</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>在代码里创建mybatis里重要入口</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">String</span> <span class="variable">resource</span> <span class="operator">=</span> <span class="string">"mybatis-config.xml"</span>;</span><br><span class="line"><span class="type">InputStream</span> <span class="variable">inputStream</span> <span class="operator">=</span> Resources.getResourceAsStream(resource);</span><br><span class="line"><span class="type">SqlSessionFactory</span> <span class="variable">sqlSessionFactory</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SqlSessionFactoryBuilder</span>().build(inputStream);</span><br></pre></td></tr></table></figure>
|
|
<p>然后我们上面的StudentMapper.xml</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="meta"><?xml version=<span class="string">"1.0"</span> encoding=<span class="string">"UTF-8"</span> ?></span></span><br><span class="line"><span class="meta"><!DOCTYPE <span class="keyword">mapper</span></span></span><br><span class="line"><span class="meta"> <span class="keyword">PUBLIC</span> <span class="string">"-//mybatis.org//DTD Mapper 3.0//EN"</span></span></span><br><span class="line"><span class="meta"> <span class="string">"https://mybatis.org/dtd/mybatis-3-mapper.dtd"</span>></span></span><br><span class="line"><span class="tag"><<span class="name">mapper</span> <span class="attr">namespace</span>=<span class="string">"com.nicksxs.mybatisdemo.StudentMapper"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">select</span> <span class="attr">id</span>=<span class="string">"selectStudent"</span> <span class="attr">resultType</span>=<span class="string">"com.nicksxs.mybatisdemo.StudentDO"</span>></span></span><br><span class="line"> select * from student where id = #{id}</span><br><span class="line"> <span class="tag"></<span class="name">select</span>></span></span><br><span class="line"><span class="tag"></<span class="name">mapper</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>那么我们就要使用这个mapper，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">String</span> <span class="variable">resource</span> <span class="operator">=</span> <span class="string">"mybatis-config.xml"</span>;</span><br><span class="line"><span class="type">InputStream</span> <span class="variable">inputStream</span> <span class="operator">=</span> Resources.getResourceAsStream(resource);</span><br><span class="line"><span class="type">SqlSessionFactory</span> <span class="variable">sqlSessionFactory</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SqlSessionFactoryBuilder</span>().build(inputStream);</span><br><span class="line"><span class="keyword">try</span> (<span class="type">SqlSession</span> <span class="variable">session</span> <span class="operator">=</span> sqlSessionFactory.openSession()) {</span><br><span class="line"> <span class="type">StudentDO</span> <span class="variable">studentDO</span> <span class="operator">=</span> session.selectOne(<span class="string">"com.nicksxs.mybatisdemo.StudentMapper.selectStudent"</span>, <span class="number">1</span>);</span><br><span class="line"> System.out.println(<span class="string">"id is "</span> + studentDO.getId() + <span class="string">" name is "</span> +studentDO.getName());</span><br><span class="line">} <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>sqlSessionFactory是sqlSession的工厂，我们可以通过sqlSessionFactory来创建sqlSession，而SqlSession 提供了在数据库执行 SQL 命令所需的所有方法。你可以通过 SqlSession 实例来直接执行已映射的 SQL 语句。可以看到mapper.xml中有定义mapper的namespace，就可以通过session.selectOne()传入namespace+id来调用这个方法<br>但是这样调用比较不合理的点，或者说按后面mybatis优化之后我们可以指定mapper接口</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">interface</span> <span class="title class_">StudentMapper</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> StudentDO <span class="title function_">selectStudent</span><span class="params">(Long id)</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>就可以可以通过mapper接口获取方法，这样就不用涉及到未知的变量转换等异常</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">try</span> (<span class="type">SqlSession</span> <span class="variable">session</span> <span class="operator">=</span> sqlSessionFactory.openSession()) {</span><br><span class="line"> <span class="type">StudentMapper</span> <span class="variable">mapper</span> <span class="operator">=</span> session.getMapper(StudentMapper.class);</span><br><span class="line"> <span class="type">StudentDO</span> <span class="variable">studentDO</span> <span class="operator">=</span> mapper.selectStudent(<span class="number">1L</span>);</span><br><span class="line"> System.out.println(<span class="string">"id is "</span> + studentDO.getId() + <span class="string">" name is "</span> +studentDO.getName());</span><br><span class="line">} <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这一篇咱们先介绍下简单的使用，后面可以先介绍下这些的原理。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Mybatis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Mysql</tag>
|
|
<tag>Mybatis</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>mybatis系列-第一条sql的更多细节</title>
|
|
<url>/2022/12/18/mybatis%E7%B3%BB%E5%88%97-%E7%AC%AC%E4%B8%80%E6%9D%A1sql%E7%9A%84%E6%9B%B4%E5%A4%9A%E7%BB%86%E8%8A%82/</url>
|
|
<content><![CDATA[<p>执行细节<br>首先设置了默认的<code>languageDriver</code><br><code>org/mybatis/mybatis/3.5.11/mybatis-3.5.11-sources.jar!/org/apache/ibatis/session/Configuration.java:215</code><br>在<code>configuration</code>的构造方法里</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">languageRegistry.setDefaultDriverClass(XMLLanguageDriver.class);</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>而在<br><code>org.apache.ibatis.builder.xml.XMLStatementBuilder#parseStatementNode </code><br>中，创建了<code>sqlSource</code>，这里就会根据前面的 <code>LanguageDriver</code> 的实现选择对应的 <code>sqlSource</code> ，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">SqlSource</span> <span class="variable">sqlSource</span> <span class="operator">=</span> langDriver.createSqlSource(configuration, context, parameterTypeClass);</span><br></pre></td></tr></table></figure>
|
|
|
|
<p><code>createSqlSource</code> 就会调用</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> SqlSource <span class="title function_">createSqlSource</span><span class="params">(Configuration configuration, XNode script, Class<?> parameterType)</span> {</span><br><span class="line"> <span class="type">XMLScriptBuilder</span> <span class="variable">builder</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">XMLScriptBuilder</span>(configuration, script, parameterType);</span><br><span class="line"> <span class="keyword">return</span> builder.parseScriptNode();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>再往下的逻辑在 <code>parseScriptNode</code> 中，<code>org.apache.ibatis.scripting.xmltags.XMLScriptBuilder#parseScriptNode</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> SqlSource <span class="title function_">parseScriptNode</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">MixedSqlNode</span> <span class="variable">rootSqlNode</span> <span class="operator">=</span> parseDynamicTags(context);</span><br><span class="line"> SqlSource sqlSource;</span><br><span class="line"> <span class="keyword">if</span> (isDynamic) {</span><br><span class="line"> sqlSource = <span class="keyword">new</span> <span class="title class_">DynamicSqlSource</span>(configuration, rootSqlNode);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> sqlSource = <span class="keyword">new</span> <span class="title class_">RawSqlSource</span>(configuration, rootSqlNode, parameterType);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> sqlSource;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>首先要解析<code>dynamicTag</code>，调用了<code>org.apache.ibatis.scripting.xmltags.XMLScriptBuilder#parseDynamicTags</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> MixedSqlNode <span class="title function_">parseDynamicTags</span><span class="params">(XNode node)</span> {</span><br><span class="line"> List<SqlNode> contents = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"> <span class="type">NodeList</span> <span class="variable">children</span> <span class="operator">=</span> node.getNode().getChildNodes();</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < children.getLength(); i++) {</span><br><span class="line"> <span class="type">XNode</span> <span class="variable">child</span> <span class="operator">=</span> node.newXNode(children.item(i));</span><br><span class="line"> <span class="keyword">if</span> (child.getNode().getNodeType() == Node.CDATA_SECTION_NODE || child.getNode().getNodeType() == Node.TEXT_NODE) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">data</span> <span class="operator">=</span> child.getStringBody(<span class="string">""</span>);</span><br><span class="line"> <span class="type">TextSqlNode</span> <span class="variable">textSqlNode</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">TextSqlNode</span>(data);</span><br><span class="line"> <span class="comment">// ---------> 主要是这边的逻辑</span></span><br><span class="line"> <span class="keyword">if</span> (textSqlNode.isDynamic()) {</span><br><span class="line"> contents.add(textSqlNode);</span><br><span class="line"> isDynamic = <span class="literal">true</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> contents.add(<span class="keyword">new</span> <span class="title class_">StaticTextSqlNode</span>(data));</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (child.getNode().getNodeType() == Node.ELEMENT_NODE) { <span class="comment">// issue #628</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">nodeName</span> <span class="operator">=</span> child.getNode().getNodeName();</span><br><span class="line"> <span class="type">NodeHandler</span> <span class="variable">handler</span> <span class="operator">=</span> nodeHandlerMap.get(nodeName);</span><br><span class="line"> <span class="keyword">if</span> (handler == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">BuilderException</span>(<span class="string">"Unknown element <"</span> + nodeName + <span class="string">"> in SQL statement."</span>);</span><br><span class="line"> }</span><br><span class="line"> handler.handleNode(child, contents);</span><br><span class="line"> isDynamic = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">MixedSqlNode</span>(contents);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>判断是否是动态<code>sql</code>，调用了<code>org.apache.ibatis.scripting.xmltags.TextSqlNode#isDynamic</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">isDynamic</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">DynamicCheckerTokenParser</span> <span class="variable">checker</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">DynamicCheckerTokenParser</span>();</span><br><span class="line"> <span class="comment">// ----------> 主要是这里的方法</span></span><br><span class="line"> <span class="type">GenericTokenParser</span> <span class="variable">parser</span> <span class="operator">=</span> createParser(checker);</span><br><span class="line"> parser.parse(text);</span><br><span class="line"> <span class="keyword">return</span> checker.isDynamic();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>创建<code>parser</code>的时候可以看到这个<code>parser</code>是干了啥，其实就是找有没有<code>${</code> , <code>}</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> GenericTokenParser <span class="title function_">createParser</span><span class="params">(TokenHandler handler)</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">GenericTokenParser</span>(<span class="string">"${"</span>, <span class="string">"}"</span>, handler);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>如果是的话，就在上面把 <code>isDynamic</code> 设置为<code>true</code> 如果是<code>true</code> 的话就创建 <code>DynamicSqlSource </code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">sqlSource = <span class="keyword">new</span> <span class="title class_">DynamicSqlSource</span>(configuration, rootSqlNode);</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>如果不是的话就创建<code>RawSqlSource</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">sqlSource = <span class="keyword">new</span> <span class="title class_">RawSqlSource</span>(configuration, rootSqlNode, parameterType);</span><br><span class="line">```java</span><br><span class="line"></span><br><span class="line">但是这不是一个真实可用的 `sqlSource` ，</span><br><span class="line">实际创建的时候会走到这</span><br><span class="line">```java</span><br><span class="line"><span class="keyword">public</span> <span class="title function_">RawSqlSource</span><span class="params">(Configuration configuration, SqlNode rootSqlNode, Class<?> parameterType)</span> {</span><br><span class="line"> <span class="built_in">this</span>(configuration, getSql(configuration, rootSqlNode), parameterType);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">RawSqlSource</span><span class="params">(Configuration configuration, String sql, Class<?> parameterType)</span> {</span><br><span class="line"> <span class="type">SqlSourceBuilder</span> <span class="variable">sqlSourceParser</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SqlSourceBuilder</span>(configuration);</span><br><span class="line"> Class<?> clazz = parameterType == <span class="literal">null</span> ? Object.class : parameterType;</span><br><span class="line"> sqlSource = sqlSourceParser.parse(sql, clazz, <span class="keyword">new</span> <span class="title class_">HashMap</span><>());</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>具体的<code>sqlSource</code>是通过<code>org.apache.ibatis.builder.SqlSourceBuilder#parse</code> 创建的<br>具体的代码逻辑是</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> SqlSource <span class="title function_">parse</span><span class="params">(String originalSql, Class<?> parameterType, Map<String, Object> additionalParameters)</span> {</span><br><span class="line"> <span class="type">ParameterMappingTokenHandler</span> <span class="variable">handler</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ParameterMappingTokenHandler</span>(configuration, parameterType, additionalParameters);</span><br><span class="line"> <span class="type">GenericTokenParser</span> <span class="variable">parser</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">GenericTokenParser</span>(<span class="string">"#{"</span>, <span class="string">"}"</span>, handler);</span><br><span class="line"> String sql;</span><br><span class="line"> <span class="keyword">if</span> (configuration.isShrinkWhitespacesInSql()) {</span><br><span class="line"> sql = parser.parse(removeExtraWhitespaces(originalSql));</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> sql = parser.parse(originalSql);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">StaticSqlSource</span>(configuration, sql, handler.getParameterMappings());</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这里创建的其实是<code>StaticSqlSource</code> ,多带一句前面的<code>parser</code>是将原来这样<code>select * from student where id = #{id}</code> 的 <code>sql</code> 解析成了<code>select * from student where id = ?</code> 然后创建了<code>StaticSqlSource</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">StaticSqlSource</span><span class="params">(Configuration configuration, String sql, List<ParameterMapping> parameterMappings)</span> {</span><br><span class="line"> <span class="built_in">this</span>.sql = sql;</span><br><span class="line"> <span class="built_in">this</span>.parameterMappings = parameterMappings;</span><br><span class="line"> <span class="built_in">this</span>.configuration = configuration;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>为什么前面要讲这么多好像没什么关系的代码呢，其实在最开始我们执行<code>sql</code>的代码中</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <E> List<E> <span class="title function_">query</span><span class="params">(MappedStatement ms, Object parameterObject, RowBounds rowBounds, ResultHandler resultHandler)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="type">BoundSql</span> <span class="variable">boundSql</span> <span class="operator">=</span> ms.getBoundSql(parameterObject);</span><br><span class="line"> <span class="type">CacheKey</span> <span class="variable">key</span> <span class="operator">=</span> createCacheKey(ms, parameterObject, rowBounds, boundSql);</span><br><span class="line"> <span class="keyword">return</span> query(ms, parameterObject, rowBounds, resultHandler, key, boundSql);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这里获取了<code>BoundSql</code>，而<code>BoundSql</code>是怎么来的呢，首先调用了<code>org.apache.ibatis.mapping.MappedStatement#getBoundSql</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> BoundSql <span class="title function_">getBoundSql</span><span class="params">(Object parameterObject)</span> {</span><br><span class="line"> <span class="type">BoundSql</span> <span class="variable">boundSql</span> <span class="operator">=</span> sqlSource.getBoundSql(parameterObject);</span><br><span class="line"> List<ParameterMapping> parameterMappings = boundSql.getParameterMappings();</span><br><span class="line"> <span class="keyword">if</span> (parameterMappings == <span class="literal">null</span> || parameterMappings.isEmpty()) {</span><br><span class="line"> boundSql = <span class="keyword">new</span> <span class="title class_">BoundSql</span>(configuration, boundSql.getSql(), parameterMap.getParameterMappings(), parameterObject);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// check for nested result maps in parameter mappings (issue #30)</span></span><br><span class="line"> <span class="keyword">for</span> (ParameterMapping pm : boundSql.getParameterMappings()) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">rmId</span> <span class="operator">=</span> pm.getResultMapId();</span><br><span class="line"> <span class="keyword">if</span> (rmId != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">ResultMap</span> <span class="variable">rm</span> <span class="operator">=</span> configuration.getResultMap(rmId);</span><br><span class="line"> <span class="keyword">if</span> (rm != <span class="literal">null</span>) {</span><br><span class="line"> hasNestedResultMaps |= rm.hasNestedResultMaps();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> boundSql;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>而我们从上面的解析中可以看到这里的<code>sqlSource</code>是一层<code>RawSqlSource</code> , 它的<code>getBoundSql</code>又是调用内部的<code>sqlSource</code>的方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> BoundSql <span class="title function_">getBoundSql</span><span class="params">(Object parameterObject)</span> {</span><br><span class="line"> <span class="keyword">return</span> sqlSource.getBoundSql(parameterObject);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>内部的<code>sqlSource</code> 就是<code>StaticSqlSource</code> ，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> BoundSql <span class="title function_">getBoundSql</span><span class="params">(Object parameterObject)</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">BoundSql</span>(configuration, sql, parameterMappings, parameterObject);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这个<code>BoundSql</code>的内容也比较简单</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">BoundSql</span><span class="params">(Configuration configuration, String sql, List<ParameterMapping> parameterMappings, Object parameterObject)</span> {</span><br><span class="line"> <span class="built_in">this</span>.sql = sql;</span><br><span class="line"> <span class="built_in">this</span>.parameterMappings = parameterMappings;</span><br><span class="line"> <span class="built_in">this</span>.parameterObject = parameterObject;</span><br><span class="line"> <span class="built_in">this</span>.additionalParameters = <span class="keyword">new</span> <span class="title class_">HashMap</span><>();</span><br><span class="line"> <span class="built_in">this</span>.metaParameters = configuration.newMetaObject(additionalParameters);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>而上次在这边<code>org.apache.ibatis.executor.SimpleExecutor#doQuery</code> 的时候落了个东西，就是<code>StatementHandler</code>的逻辑</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <E> List<E> <span class="title function_">doQuery</span><span class="params">(MappedStatement ms, Object parameter, RowBounds rowBounds, ResultHandler resultHandler, BoundSql boundSql)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="type">Statement</span> <span class="variable">stmt</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">Configuration</span> <span class="variable">configuration</span> <span class="operator">=</span> ms.getConfiguration();</span><br><span class="line"> <span class="type">StatementHandler</span> <span class="variable">handler</span> <span class="operator">=</span> configuration.newStatementHandler(wrapper, ms, parameter, rowBounds, resultHandler, boundSql);</span><br><span class="line"> stmt = prepareStatement(handler, ms.getStatementLog());</span><br><span class="line"> <span class="keyword">return</span> handler.query(stmt, resultHandler);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> closeStatement(stmt);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>它是通过statementType来区分应该使用哪个statementHandler，我们这使用的就是PreparedStatementHandler </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">RoutingStatementHandler</span><span class="params">(Executor executor, MappedStatement ms, Object parameter, RowBounds rowBounds, ResultHandler resultHandler, BoundSql boundSql)</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">switch</span> (ms.getStatementType()) {</span><br><span class="line"> <span class="keyword">case</span> STATEMENT:</span><br><span class="line"> delegate = <span class="keyword">new</span> <span class="title class_">SimpleStatementHandler</span>(executor, ms, parameter, rowBounds, resultHandler, boundSql);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">case</span> PREPARED:</span><br><span class="line"> delegate = <span class="keyword">new</span> <span class="title class_">PreparedStatementHandler</span>(executor, ms, parameter, rowBounds, resultHandler, boundSql);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">case</span> CALLABLE:</span><br><span class="line"> delegate = <span class="keyword">new</span> <span class="title class_">CallableStatementHandler</span>(executor, ms, parameter, rowBounds, resultHandler, boundSql);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">default</span>:</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">ExecutorException</span>(<span class="string">"Unknown statement type: "</span> + ms.getStatementType());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>所以上次有个细节可以补充，这边的doQuery里面的handler.query 应该是调用了PreparedStatementHandler 的query方法 </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <E> List<E> <span class="title function_">doQuery</span><span class="params">(MappedStatement ms, Object parameter, RowBounds rowBounds, ResultHandler resultHandler, BoundSql boundSql)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="type">Statement</span> <span class="variable">stmt</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">Configuration</span> <span class="variable">configuration</span> <span class="operator">=</span> ms.getConfiguration();</span><br><span class="line"> <span class="type">StatementHandler</span> <span class="variable">handler</span> <span class="operator">=</span> configuration.newStatementHandler(wrapper, ms, parameter, rowBounds, resultHandler, boundSql);</span><br><span class="line"> stmt = prepareStatement(handler, ms.getStatementLog());</span><br><span class="line"> <span class="keyword">return</span> handler.query(stmt, resultHandler);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> closeStatement(stmt);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
<p>因为上面prepareStatement中getConnection拿到connection是com.mysql.cj.jdbc.ConnectionImpl#ConnectionImpl(com.mysql.cj.conf.HostInfo) </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <E> List<E> <span class="title function_">query</span><span class="params">(Statement statement, ResultHandler resultHandler)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="type">PreparedStatement</span> <span class="variable">ps</span> <span class="operator">=</span> (PreparedStatement) statement;</span><br><span class="line"> ps.execute();</span><br><span class="line"> <span class="keyword">return</span> resultSetHandler.handleResultSets(ps);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>那又为什么是这个呢，可以在网上找，我们在mybatis-config.xml里配置的</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">transactionManager</span> <span class="attr">type</span>=<span class="string">"JDBC"</span>/></span></span><br></pre></td></tr></table></figure>
|
|
|
|
<p>因此在parseConfiguration中配置environment时</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">parseConfiguration</span><span class="params">(XNode root)</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// issue #117 read properties first</span></span><br><span class="line"> propertiesElement(root.evalNode(<span class="string">"properties"</span>));</span><br><span class="line"> <span class="type">Properties</span> <span class="variable">settings</span> <span class="operator">=</span> settingsAsProperties(root.evalNode(<span class="string">"settings"</span>));</span><br><span class="line"> loadCustomVfs(settings);</span><br><span class="line"> loadCustomLogImpl(settings);</span><br><span class="line"> typeAliasesElement(root.evalNode(<span class="string">"typeAliases"</span>));</span><br><span class="line"> pluginElement(root.evalNode(<span class="string">"plugins"</span>));</span><br><span class="line"> objectFactoryElement(root.evalNode(<span class="string">"objectFactory"</span>));</span><br><span class="line"> objectWrapperFactoryElement(root.evalNode(<span class="string">"objectWrapperFactory"</span>));</span><br><span class="line"> reflectorFactoryElement(root.evalNode(<span class="string">"reflectorFactory"</span>));</span><br><span class="line"> settingsElement(settings);</span><br><span class="line"> <span class="comment">// read it after objectFactory and objectWrapperFactory issue #631</span></span><br><span class="line"> <span class="comment">// ----------> 就是这里</span></span><br><span class="line"> environmentsElement(root.evalNode(<span class="string">"environments"</span>));</span><br><span class="line"> databaseIdProviderElement(root.evalNode(<span class="string">"databaseIdProvider"</span>));</span><br><span class="line"> typeHandlerElement(root.evalNode(<span class="string">"typeHandlers"</span>));</span><br><span class="line"> mapperElement(root.evalNode(<span class="string">"mappers"</span>));</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">BuilderException</span>(<span class="string">"Error parsing SQL Mapper Configuration. Cause: "</span> + e, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>调用的这个方法通过获取xml中的transactionManager 配置的类型，也就是JDBC </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">environmentsElement</span><span class="params">(XNode context)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="keyword">if</span> (context != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">if</span> (environment == <span class="literal">null</span>) {</span><br><span class="line"> environment = context.getStringAttribute(<span class="string">"default"</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">for</span> (XNode child : context.getChildren()) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">id</span> <span class="operator">=</span> child.getStringAttribute(<span class="string">"id"</span>);</span><br><span class="line"> <span class="keyword">if</span> (isSpecifiedEnvironment(id)) {</span><br><span class="line"> <span class="comment">// -------> 找到这里</span></span><br><span class="line"> <span class="type">TransactionFactory</span> <span class="variable">txFactory</span> <span class="operator">=</span> transactionManagerElement(child.evalNode(<span class="string">"transactionManager"</span>));</span><br><span class="line"> <span class="type">DataSourceFactory</span> <span class="variable">dsFactory</span> <span class="operator">=</span> dataSourceElement(child.evalNode(<span class="string">"dataSource"</span>));</span><br><span class="line"> <span class="type">DataSource</span> <span class="variable">dataSource</span> <span class="operator">=</span> dsFactory.getDataSource();</span><br><span class="line"> Environment.<span class="type">Builder</span> <span class="variable">environmentBuilder</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Environment</span>.Builder(id)</span><br><span class="line"> .transactionFactory(txFactory)</span><br><span class="line"> .dataSource(dataSource);</span><br><span class="line"> configuration.setEnvironment(environmentBuilder.build());</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>是通过以下方法获取的，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// 方法全限定名 org.apache.ibatis.builder.xml.XMLConfigBuilder#transactionManagerElement</span></span><br><span class="line"><span class="keyword">private</span> TransactionFactory <span class="title function_">transactionManagerElement</span><span class="params">(XNode context)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="keyword">if</span> (context != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">type</span> <span class="operator">=</span> context.getStringAttribute(<span class="string">"type"</span>);</span><br><span class="line"> <span class="type">Properties</span> <span class="variable">props</span> <span class="operator">=</span> context.getChildrenAsProperties();</span><br><span class="line"> <span class="type">TransactionFactory</span> <span class="variable">factory</span> <span class="operator">=</span> (TransactionFactory) resolveClass(type).getDeclaredConstructor().newInstance();</span><br><span class="line"> factory.setProperties(props);</span><br><span class="line"> <span class="keyword">return</span> factory;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">BuilderException</span>(<span class="string">"Environment declaration requires a TransactionFactory."</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"><span class="comment">// 方法全限定名 org.apache.ibatis.builder.BaseBuilder#resolveClass</span></span><br><span class="line"><span class="keyword">protected</span> <T> Class<? <span class="keyword">extends</span> <span class="title class_">T</span>> resolveClass(String alias) {</span><br><span class="line"> <span class="keyword">if</span> (alias == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">return</span> resolveAlias(alias);</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">BuilderException</span>(<span class="string">"Error resolving class. Cause: "</span> + e, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"><span class="comment">// 方法全限定名 org.apache.ibatis.builder.BaseBuilder#resolveAlias</span></span><br><span class="line"> <span class="keyword">protected</span> <T> Class<? <span class="keyword">extends</span> <span class="title class_">T</span>> resolveAlias(String alias) {</span><br><span class="line"> <span class="keyword">return</span> typeAliasRegistry.resolveAlias(alias);</span><br><span class="line"> }</span><br><span class="line"><span class="comment">// 方法全限定名 org.apache.ibatis.type.TypeAliasRegistry#resolveAlias</span></span><br><span class="line"> <span class="keyword">public</span> <T> Class<T> <span class="title function_">resolveAlias</span><span class="params">(String string)</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (string == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// issue #748</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">key</span> <span class="operator">=</span> string.toLowerCase(Locale.ENGLISH);</span><br><span class="line"> Class<T> value;</span><br><span class="line"> <span class="keyword">if</span> (typeAliases.containsKey(key)) {</span><br><span class="line"> value = (Class<T>) typeAliases.get(key);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> value = (Class<T>) Resources.classForName(string);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> value;</span><br><span class="line"> } <span class="keyword">catch</span> (ClassNotFoundException e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">TypeException</span>(<span class="string">"Could not resolve type alias '"</span> + string + <span class="string">"'. Cause: "</span> + e, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>而通过JDBC获取得是啥的，就是在Configuration的构造方法里写了的JdbcTransactionFactory </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">Configuration</span><span class="params">()</span> {</span><br><span class="line"> typeAliasRegistry.registerAlias(<span class="string">"JDBC"</span>, JdbcTransactionFactory.class);</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>所以我们在这</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> SqlSession <span class="title function_">openSessionFromDataSource</span><span class="params">(ExecutorType execType, TransactionIsolationLevel level, <span class="type">boolean</span> autoCommit)</span> {</span><br><span class="line"> <span class="type">Transaction</span> <span class="variable">tx</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">Environment</span> <span class="variable">environment</span> <span class="operator">=</span> configuration.getEnvironment();</span><br><span class="line"> <span class="keyword">final</span> <span class="type">TransactionFactory</span> <span class="variable">transactionFactory</span> <span class="operator">=</span> getTransactionFactoryFromEnvironment(environment);</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>获得到的TransactionFactory 就是 JdbcTransactionFactory ，而后</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">tx = transactionFactory.newTransaction(environment.getDataSource(), level, autoCommit);</span><br><span class="line">```java</span><br><span class="line"></span><br><span class="line">创建的transaction就是JdbcTransaction </span><br><span class="line">```java</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> Transaction <span class="title function_">newTransaction</span><span class="params">(DataSource ds, TransactionIsolationLevel level, <span class="type">boolean</span> autoCommit)</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">JdbcTransaction</span>(ds, level, autoCommit, skipSetAutoCommitOnClose);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后我们再会上去看代码getConnection ,</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> Connection <span class="title function_">getConnection</span><span class="params">(Log statementLog)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="comment">// -------> 这里的transaction就是JdbcTransaction</span></span><br><span class="line"> <span class="type">Connection</span> <span class="variable">connection</span> <span class="operator">=</span> transaction.getConnection();</span><br><span class="line"> <span class="keyword">if</span> (statementLog.isDebugEnabled()) {</span><br><span class="line"> <span class="keyword">return</span> ConnectionLogger.newInstance(connection, statementLog, queryStack);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> connection;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>即调用了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> Connection <span class="title function_">getConnection</span><span class="params">()</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="keyword">if</span> (connection == <span class="literal">null</span>) {</span><br><span class="line"> openConnection();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> connection;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">openConnection</span><span class="params">()</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"Opening JDBC Connection"</span>);</span><br><span class="line"> }</span><br><span class="line"> connection = dataSource.getConnection();</span><br><span class="line"> <span class="keyword">if</span> (level != <span class="literal">null</span>) {</span><br><span class="line"> connection.setTransactionIsolation(level.getLevel());</span><br><span class="line"> }</span><br><span class="line"> setDesiredAutoCommit(autoCommit);</span><br><span class="line"> }</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> Connection <span class="title function_">getConnection</span><span class="params">()</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="keyword">return</span> popConnection(dataSource.getUsername(), dataSource.getPassword()).getProxyConnection();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"><span class="keyword">private</span> PooledConnection <span class="title function_">popConnection</span><span class="params">(String username, String password)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">countedWait</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="type">PooledConnection</span> <span class="variable">conn</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="type">long</span> <span class="variable">t</span> <span class="operator">=</span> System.currentTimeMillis();</span><br><span class="line"> <span class="type">int</span> <span class="variable">localBadConnectionCount</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">while</span> (conn == <span class="literal">null</span>) {</span><br><span class="line"> lock.lock();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (!state.idleConnections.isEmpty()) {</span><br><span class="line"> <span class="comment">// Pool has available connection</span></span><br><span class="line"> conn = state.idleConnections.remove(<span class="number">0</span>);</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"Checked out connection "</span> + conn.getRealHashCode() + <span class="string">" from pool."</span>);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// Pool does not have available connection</span></span><br><span class="line"> <span class="keyword">if</span> (state.activeConnections.size() < poolMaximumActiveConnections) {</span><br><span class="line"> <span class="comment">// Can create new connection</span></span><br><span class="line"> <span class="comment">// ------------> 走到这里会创建PooledConnection，但是里面会先调用dataSource.getConnection()</span></span><br><span class="line"> conn = <span class="keyword">new</span> <span class="title class_">PooledConnection</span>(dataSource.getConnection(), <span class="built_in">this</span>);</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"Created connection "</span> + conn.getRealHashCode() + <span class="string">"."</span>);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// Cannot create new connection</span></span><br><span class="line"> <span class="type">PooledConnection</span> <span class="variable">oldestActiveConnection</span> <span class="operator">=</span> state.activeConnections.get(<span class="number">0</span>);</span><br><span class="line"> <span class="type">long</span> <span class="variable">longestCheckoutTime</span> <span class="operator">=</span> oldestActiveConnection.getCheckoutTime();</span><br><span class="line"> <span class="keyword">if</span> (longestCheckoutTime > poolMaximumCheckoutTime) {</span><br><span class="line"> <span class="comment">// Can claim overdue connection</span></span><br><span class="line"> state.claimedOverdueConnectionCount++;</span><br><span class="line"> state.accumulatedCheckoutTimeOfOverdueConnections += longestCheckoutTime;</span><br><span class="line"> state.accumulatedCheckoutTime += longestCheckoutTime;</span><br><span class="line"> state.activeConnections.remove(oldestActiveConnection);</span><br><span class="line"> <span class="keyword">if</span> (!oldestActiveConnection.getRealConnection().getAutoCommit()) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> oldestActiveConnection.getRealConnection().rollback();</span><br><span class="line"> } <span class="keyword">catch</span> (SQLException e) {</span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> Just log a message for debug and continue to execute the following</span></span><br><span class="line"><span class="comment"> statement like nothing happened.</span></span><br><span class="line"><span class="comment"> Wrap the bad connection with a new PooledConnection, this will help</span></span><br><span class="line"><span class="comment"> to not interrupt current executing thread and give current thread a</span></span><br><span class="line"><span class="comment"> chance to join the next competition for another valid/good database</span></span><br><span class="line"><span class="comment"> connection. At the end of this loop, bad {@link @conn} will be set as null.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> log.debug(<span class="string">"Bad connection. Could not roll back"</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> conn = <span class="keyword">new</span> <span class="title class_">PooledConnection</span>(oldestActiveConnection.getRealConnection(), <span class="built_in">this</span>);</span><br><span class="line"> conn.setCreatedTimestamp(oldestActiveConnection.getCreatedTimestamp());</span><br><span class="line"> conn.setLastUsedTimestamp(oldestActiveConnection.getLastUsedTimestamp());</span><br><span class="line"> oldestActiveConnection.invalidate();</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"Claimed overdue connection "</span> + conn.getRealHashCode() + <span class="string">"."</span>);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// Must wait</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (!countedWait) {</span><br><span class="line"> state.hadToWaitCount++;</span><br><span class="line"> countedWait = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"Waiting as long as "</span> + poolTimeToWait + <span class="string">" milliseconds for connection."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="type">long</span> <span class="variable">wt</span> <span class="operator">=</span> System.currentTimeMillis();</span><br><span class="line"> condition.await(poolTimeToWait, TimeUnit.MILLISECONDS);</span><br><span class="line"> state.accumulatedWaitTime += System.currentTimeMillis() - wt;</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> <span class="comment">// set interrupt flag</span></span><br><span class="line"> Thread.currentThread().interrupt();</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (conn != <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// ping to server and check the connection is valid or not</span></span><br><span class="line"> <span class="keyword">if</span> (conn.isValid()) {</span><br><span class="line"> <span class="keyword">if</span> (!conn.getRealConnection().getAutoCommit()) {</span><br><span class="line"> conn.getRealConnection().rollback();</span><br><span class="line"> }</span><br><span class="line"> conn.setConnectionTypeCode(assembleConnectionTypeCode(dataSource.getUrl(), username, password));</span><br><span class="line"> conn.setCheckoutTimestamp(System.currentTimeMillis());</span><br><span class="line"> conn.setLastUsedTimestamp(System.currentTimeMillis());</span><br><span class="line"> state.activeConnections.add(conn);</span><br><span class="line"> state.requestCount++;</span><br><span class="line"> state.accumulatedRequestTime += System.currentTimeMillis() - t;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"A bad connection ("</span> + conn.getRealHashCode() + <span class="string">") was returned from the pool, getting another connection."</span>);</span><br><span class="line"> }</span><br><span class="line"> state.badConnectionCount++;</span><br><span class="line"> localBadConnectionCount++;</span><br><span class="line"> conn = <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">if</span> (localBadConnectionCount > (poolMaximumIdleConnections + poolMaximumLocalBadConnectionTolerance)) {</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"PooledDataSource: Could not get a good connection to the database."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">SQLException</span>(<span class="string">"PooledDataSource: Could not get a good connection to the database."</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> lock.unlock();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (conn == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">if</span> (log.isDebugEnabled()) {</span><br><span class="line"> log.debug(<span class="string">"PooledDataSource: Unknown severe error condition. The connection pool returned a null connection."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">SQLException</span>(<span class="string">"PooledDataSource: Unknown severe error condition. The connection pool returned a null connection."</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> conn;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>其实就是调用的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// org.apache.ibatis.datasource.unpooled.UnpooledDataSource#getConnection()</span></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> Connection <span class="title function_">getConnection</span><span class="params">()</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="keyword">return</span> doGetConnection(username, password);</span><br><span class="line"> }</span><br><span class="line">```java</span><br><span class="line"></span><br><span class="line">然后就是</span><br><span class="line">```java</span><br><span class="line"><span class="keyword">private</span> Connection <span class="title function_">doGetConnection</span><span class="params">(String username, String password)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="type">Properties</span> <span class="variable">props</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Properties</span>();</span><br><span class="line"> <span class="keyword">if</span> (driverProperties != <span class="literal">null</span>) {</span><br><span class="line"> props.putAll(driverProperties);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (username != <span class="literal">null</span>) {</span><br><span class="line"> props.setProperty(<span class="string">"user"</span>, username);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (password != <span class="literal">null</span>) {</span><br><span class="line"> props.setProperty(<span class="string">"password"</span>, password);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> doGetConnection(props);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>继续这个逻辑</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"> <span class="keyword">private</span> Connection <span class="title function_">doGetConnection</span><span class="params">(Properties properties)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> initializeDriver();</span><br><span class="line"> <span class="type">Connection</span> <span class="variable">connection</span> <span class="operator">=</span> DriverManager.getConnection(url, properties);</span><br><span class="line"> configureConnection(connection);</span><br><span class="line"> <span class="keyword">return</span> connection;</span><br><span class="line"> }</span><br><span class="line"> <span class="meta">@CallerSensitive</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> Connection <span class="title function_">getConnection</span><span class="params">(String url,</span></span><br><span class="line"><span class="params"> java.util.Properties info)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> (getConnection(url, info, Reflection.getCallerClass()));</span><br><span class="line"> }</span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> Connection <span class="title function_">getConnection</span><span class="params">(</span></span><br><span class="line"><span class="params"> String url, java.util.Properties info, Class<?> caller)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> * When callerCl is null, we should check the application's</span></span><br><span class="line"><span class="comment"> * (which is invoking this class indirectly)</span></span><br><span class="line"><span class="comment"> * classloader, so that the JDBC driver class outside rt.jar</span></span><br><span class="line"><span class="comment"> * can be loaded from here.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="type">ClassLoader</span> <span class="variable">callerCL</span> <span class="operator">=</span> caller != <span class="literal">null</span> ? caller.getClassLoader() : <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">synchronized</span>(DriverManager.class) {</span><br><span class="line"> <span class="comment">// synchronize loading of the correct classloader.</span></span><br><span class="line"> <span class="keyword">if</span> (callerCL == <span class="literal">null</span>) {</span><br><span class="line"> callerCL = Thread.currentThread().getContextClassLoader();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span>(url == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">SQLException</span>(<span class="string">"The url cannot be null"</span>, <span class="string">"08001"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> println(<span class="string">"DriverManager.getConnection(\""</span> + url + <span class="string">"\")"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Walk through the loaded registeredDrivers attempting to make a connection.</span></span><br><span class="line"> <span class="comment">// Remember the first exception that gets raised so we can reraise it.</span></span><br><span class="line"> <span class="type">SQLException</span> <span class="variable">reason</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span>(DriverInfo aDriver : registeredDrivers) {</span><br><span class="line"> <span class="comment">// If the caller does not have permission to load the driver then</span></span><br><span class="line"> <span class="comment">// skip it.</span></span><br><span class="line"> <span class="keyword">if</span>(isDriverAllowed(aDriver.driver, callerCL)) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// ----------> driver[className=com.mysql.cj.jdbc.Driver@64030b91]</span></span><br><span class="line"> println(<span class="string">" trying "</span> + aDriver.driver.getClass().getName());</span><br><span class="line"> <span class="type">Connection</span> <span class="variable">con</span> <span class="operator">=</span> aDriver.driver.connect(url, info);</span><br><span class="line"> <span class="keyword">if</span> (con != <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// Success!</span></span><br><span class="line"> println(<span class="string">"getConnection returning "</span> + aDriver.driver.getClass().getName());</span><br><span class="line"> <span class="keyword">return</span> (con);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (SQLException ex) {</span><br><span class="line"> <span class="keyword">if</span> (reason == <span class="literal">null</span>) {</span><br><span class="line"> reason = ex;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> println(<span class="string">" skipping: "</span> + aDriver.getClass().getName());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// if we got here nobody could connect.</span></span><br><span class="line"> <span class="keyword">if</span> (reason != <span class="literal">null</span>) {</span><br><span class="line"> println(<span class="string">"getConnection failed: "</span> + reason);</span><br><span class="line"> <span class="keyword">throw</span> reason;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> println(<span class="string">"getConnection: no suitable driver found for "</span>+ url);</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">SQLException</span>(<span class="string">"No suitable driver found for "</span>+ url, <span class="string">"08001"</span>);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
<p>上面的driver就是driver[className=com.mysql.cj.jdbc.Driver@64030b91]</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// com.mysql.cj.jdbc.NonRegisteringDriver#connect</span></span><br><span class="line"><span class="keyword">public</span> Connection <span class="title function_">connect</span><span class="params">(String url, Properties info)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (!ConnectionUrl.acceptsUrl(url)) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="type">ConnectionUrl</span> <span class="variable">conStr</span> <span class="operator">=</span> ConnectionUrl.getConnectionUrlInstance(url, info);</span><br><span class="line"> <span class="keyword">switch</span> (conStr.getType()) {</span><br><span class="line"> <span class="keyword">case</span> SINGLE_CONNECTION:</span><br><span class="line"> <span class="keyword">return</span> ConnectionImpl.getInstance(conStr.getMainHost());</span><br><span class="line"> <span class="keyword">case</span> FAILOVER_CONNECTION:</span><br><span class="line"> <span class="keyword">case</span> FAILOVER_DNS_SRV_CONNECTION:</span><br><span class="line"> <span class="keyword">return</span> FailoverConnectionProxy.createProxyInstance(conStr);</span><br><span class="line"> <span class="keyword">case</span> LOADBALANCE_CONNECTION:</span><br><span class="line"> <span class="keyword">case</span> LOADBALANCE_DNS_SRV_CONNECTION:</span><br><span class="line"> <span class="keyword">return</span> LoadBalancedConnectionProxy.createProxyInstance(conStr);</span><br><span class="line"> <span class="keyword">case</span> REPLICATION_CONNECTION:</span><br><span class="line"> <span class="keyword">case</span> REPLICATION_DNS_SRV_CONNECTION:</span><br><span class="line"> <span class="keyword">return</span> ReplicationConnectionProxy.createProxyInstance(conStr);</span><br><span class="line"> <span class="keyword">default</span>:</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (UnsupportedConnectionStringException var5) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> } <span class="keyword">catch</span> (CJException var6) {</span><br><span class="line"> <span class="keyword">throw</span> (UnableToConnectException)ExceptionFactory.createException(UnableToConnectException.class, Messages.getString(<span class="string">"NonRegisteringDriver.17"</span>, <span class="keyword">new</span> <span class="title class_">Object</span>[]{var6.toString()}), var6);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (CJException var7) {</span><br><span class="line"> <span class="keyword">throw</span> SQLExceptionsMapping.translateException(var7);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这是个 SINGLE_CONNECTION ,所以调用的就是 return ConnectionImpl.getInstance(conStr.getMainHost());<br>然后在这里设置了代理类</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">PooledConnection</span><span class="params">(Connection connection, PooledDataSource dataSource)</span> {</span><br><span class="line"> <span class="built_in">this</span>.hashCode = connection.hashCode();</span><br><span class="line"> <span class="built_in">this</span>.realConnection = connection;</span><br><span class="line"> <span class="built_in">this</span>.dataSource = dataSource;</span><br><span class="line"> <span class="built_in">this</span>.createdTimestamp = System.currentTimeMillis();</span><br><span class="line"> <span class="built_in">this</span>.lastUsedTimestamp = System.currentTimeMillis();</span><br><span class="line"> <span class="built_in">this</span>.valid = <span class="literal">true</span>;</span><br><span class="line"> <span class="built_in">this</span>.proxyConnection = (Connection) Proxy.newProxyInstance(Connection.class.getClassLoader(), IFACES, <span class="built_in">this</span>);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>结合这个</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> Connection <span class="title function_">getConnection</span><span class="params">()</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="keyword">return</span> popConnection(dataSource.getUsername(), dataSource.getPassword()).getProxyConnection();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>所以最终的connection就是com.mysql.cj.jdbc.ConnectionImpl@358ab600</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Mybatis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Mysql</tag>
|
|
<tag>Mybatis</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>mybatis系列-第一条sql的细节</title>
|
|
<url>/2022/12/11/mybatis%E7%B3%BB%E5%88%97-%E7%AC%AC%E4%B8%80%E6%9D%A1sql%E7%9A%84%E7%BB%86%E8%8A%82/</url>
|
|
<content><![CDATA[<p>先补充两个点，<br>第一是前面我们说了<br>使用<code>org.apache.ibatis.builder.xml.XMLConfigBuilder</code> 创建了<code>parser</code>解析器，那么解析的结果是什么<br>看这个方法的返回值</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Configuration <span class="title function_">parse</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (parsed) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">BuilderException</span>(<span class="string">"Each XMLConfigBuilder can only be used once."</span>);</span><br><span class="line"> }</span><br><span class="line"> parsed = <span class="literal">true</span>;</span><br><span class="line"> parseConfiguration(parser.evalNode(<span class="string">"/configuration"</span>));</span><br><span class="line"> <span class="keyword">return</span> configuration;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>返回的是 <code>org.apache.ibatis.session.Configuration</code> , 而这个 <code>Configuration</code> 也是 <code>mybatis</code> 中特别重要的配置核心类，贴一下里面的成员变量，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">Configuration</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> Environment environment;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="type">boolean</span> safeRowBoundsEnabled;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">boolean</span> <span class="variable">safeResultHandlerEnabled</span> <span class="operator">=</span> <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">boolean</span> mapUnderscoreToCamelCase;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">boolean</span> aggressiveLazyLoading;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">boolean</span> <span class="variable">multipleResultSetsEnabled</span> <span class="operator">=</span> <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">boolean</span> useGeneratedKeys;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">boolean</span> <span class="variable">useColumnLabel</span> <span class="operator">=</span> <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">boolean</span> <span class="variable">cacheEnabled</span> <span class="operator">=</span> <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">boolean</span> callSettersOnNulls;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">boolean</span> <span class="variable">useActualParamName</span> <span class="operator">=</span> <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">boolean</span> returnInstanceForEmptyRow;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">boolean</span> shrinkWhitespacesInSql;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">boolean</span> nullableOnForEach;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">boolean</span> argNameBasedConstructorAutoMapping;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> String logPrefix;</span><br><span class="line"> <span class="keyword">protected</span> Class<? <span class="keyword">extends</span> <span class="title class_">Log</span>> logImpl;</span><br><span class="line"> <span class="keyword">protected</span> Class<? <span class="keyword">extends</span> <span class="title class_">VFS</span>> vfsImpl;</span><br><span class="line"> <span class="keyword">protected</span> Class<?> defaultSqlProviderType;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">LocalCacheScope</span> <span class="variable">localCacheScope</span> <span class="operator">=</span> LocalCacheScope.SESSION;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">JdbcType</span> <span class="variable">jdbcTypeForNull</span> <span class="operator">=</span> JdbcType.OTHER;</span><br><span class="line"> <span class="keyword">protected</span> Set<String> lazyLoadTriggerMethods = <span class="keyword">new</span> <span class="title class_">HashSet</span><>(Arrays.asList(<span class="string">"equals"</span>, <span class="string">"clone"</span>, <span class="string">"hashCode"</span>, <span class="string">"toString"</span>));</span><br><span class="line"> <span class="keyword">protected</span> Integer defaultStatementTimeout;</span><br><span class="line"> <span class="keyword">protected</span> Integer defaultFetchSize;</span><br><span class="line"> <span class="keyword">protected</span> ResultSetType defaultResultSetType;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">ExecutorType</span> <span class="variable">defaultExecutorType</span> <span class="operator">=</span> ExecutorType.SIMPLE;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">AutoMappingBehavior</span> <span class="variable">autoMappingBehavior</span> <span class="operator">=</span> AutoMappingBehavior.PARTIAL;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">AutoMappingUnknownColumnBehavior</span> <span class="variable">autoMappingUnknownColumnBehavior</span> <span class="operator">=</span> AutoMappingUnknownColumnBehavior.NONE;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="type">Properties</span> <span class="variable">variables</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Properties</span>();</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">ReflectorFactory</span> <span class="variable">reflectorFactory</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">DefaultReflectorFactory</span>();</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">ObjectFactory</span> <span class="variable">objectFactory</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">DefaultObjectFactory</span>();</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">ObjectWrapperFactory</span> <span class="variable">objectWrapperFactory</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">DefaultObjectWrapperFactory</span>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="type">boolean</span> <span class="variable">lazyLoadingEnabled</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="keyword">protected</span> <span class="type">ProxyFactory</span> <span class="variable">proxyFactory</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">JavassistProxyFactory</span>(); <span class="comment">// #224 Using internal Javassist instead of OGNL</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> String databaseId;</span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Configuration factory class.</span></span><br><span class="line"><span class="comment"> * Used to create Configuration for loading deserialized unread properties.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> <a href='https://github.com/mybatis/old-google-code-issues/issues/300'>Issue 300 (google code)</a></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">protected</span> Class<?> configurationFactory;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> <span class="type">MapperRegistry</span> <span class="variable">mapperRegistry</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">MapperRegistry</span>(<span class="built_in">this</span>);</span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> <span class="type">InterceptorChain</span> <span class="variable">interceptorChain</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">InterceptorChain</span>();</span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> <span class="type">TypeHandlerRegistry</span> <span class="variable">typeHandlerRegistry</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">TypeHandlerRegistry</span>(<span class="built_in">this</span>);</span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> <span class="type">TypeAliasRegistry</span> <span class="variable">typeAliasRegistry</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">TypeAliasRegistry</span>();</span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> <span class="type">LanguageDriverRegistry</span> <span class="variable">languageRegistry</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">LanguageDriverRegistry</span>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> Map<String, MappedStatement> mappedStatements = <span class="keyword">new</span> <span class="title class_">StrictMap</span><MappedStatement>(<span class="string">"Mapped Statements collection"</span>)</span><br><span class="line"> .conflictMessageProducer((savedValue, targetValue) -></span><br><span class="line"> <span class="string">". please check "</span> + savedValue.getResource() + <span class="string">" and "</span> + targetValue.getResource());</span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> Map<String, Cache> caches = <span class="keyword">new</span> <span class="title class_">StrictMap</span><>(<span class="string">"Caches collection"</span>);</span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> Map<String, ResultMap> resultMaps = <span class="keyword">new</span> <span class="title class_">StrictMap</span><>(<span class="string">"Result Maps collection"</span>);</span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> Map<String, ParameterMap> parameterMaps = <span class="keyword">new</span> <span class="title class_">StrictMap</span><>(<span class="string">"Parameter Maps collection"</span>);</span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> Map<String, KeyGenerator> keyGenerators = <span class="keyword">new</span> <span class="title class_">StrictMap</span><>(<span class="string">"Key Generators collection"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> Set<String> loadedResources = <span class="keyword">new</span> <span class="title class_">HashSet</span><>();</span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> Map<String, XNode> sqlFragments = <span class="keyword">new</span> <span class="title class_">StrictMap</span><>(<span class="string">"XML fragments parsed from previous mappers"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> Collection<XMLStatementBuilder> incompleteStatements = <span class="keyword">new</span> <span class="title class_">LinkedList</span><>();</span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> Collection<CacheRefResolver> incompleteCacheRefs = <span class="keyword">new</span> <span class="title class_">LinkedList</span><>();</span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> Collection<ResultMapResolver> incompleteResultMaps = <span class="keyword">new</span> <span class="title class_">LinkedList</span><>();</span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> Collection<MethodResolver> incompleteMethods = <span class="keyword">new</span> <span class="title class_">LinkedList</span><>();</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这么多成员变量，先不一一解释作用，但是其中的几个参数我们应该是已经知道了的，第一个就是 <code>mappedStatements</code> ，上一篇我们知道被解析的mapper就是放在这里，后面的 <code>resultMaps</code> ，<code>parameterMaps</code> 也比较常用的就是我们参数和结果的映射<code>map</code>，这里跟我之前有一篇解释为啥我们一些变量的使用会比较特殊，比如<code>list</code>，可以<a href="https://nicksxs.me/2022/07/09/mybatis-%E7%9A%84-foreach-%E4%BD%BF%E7%94%A8%E7%9A%84%E6%B3%A8%E6%84%8F%E7%82%B9/">参考这篇</a>，<code>keyGenerators</code>是在我们需要定义主键生成器的时候使用。<br>然后第二点是我们创建的 <code>org.apache.ibatis.session.SqlSessionFactory</code> 是哪个，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> SqlSessionFactory <span class="title function_">build</span><span class="params">(Configuration config)</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">DefaultSqlSessionFactory</span>(config);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>是这个 <code>DefaultSqlSessionFactory</code> ，这是其中一个 <code>SqlSessionFactory</code> 的实现<br>接下来我们看看 <code>openSession</code> 里干了啥</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> SqlSession <span class="title function_">openSession</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> openSessionFromDataSource(configuration.getDefaultExecutorType(), <span class="literal">null</span>, <span class="literal">false</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这边有几个参数，第一个是默认的执行器类型，往上找找上面贴着的 <code>Configuration</code> 的成员变量里可以看到默认是<br><code>protected ExecutorType defaultExecutorType = ExecutorType.SIMPLE;</code></p>
|
|
<p>因为没有指明特殊的执行逻辑，所以默认我们也就用简单类型的，第二个参数是是事务级别，第三个是是否自动提交</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> SqlSession <span class="title function_">openSessionFromDataSource</span><span class="params">(ExecutorType execType, TransactionIsolationLevel level, <span class="type">boolean</span> autoCommit)</span> {</span><br><span class="line"> <span class="type">Transaction</span> <span class="variable">tx</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">Environment</span> <span class="variable">environment</span> <span class="operator">=</span> configuration.getEnvironment();</span><br><span class="line"> <span class="keyword">final</span> <span class="type">TransactionFactory</span> <span class="variable">transactionFactory</span> <span class="operator">=</span> getTransactionFactoryFromEnvironment(environment);</span><br><span class="line"> tx = transactionFactory.newTransaction(environment.getDataSource(), level, autoCommit);</span><br><span class="line"> <span class="comment">// --------> 先关注这里</span></span><br><span class="line"> <span class="keyword">final</span> <span class="type">Executor</span> <span class="variable">executor</span> <span class="operator">=</span> configuration.newExecutor(tx, execType);</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">DefaultSqlSession</span>(configuration, executor, autoCommit);</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> closeTransaction(tx); <span class="comment">// may have fetched a connection so lets call close()</span></span><br><span class="line"> <span class="keyword">throw</span> ExceptionFactory.wrapException(<span class="string">"Error opening session. Cause: "</span> + e, e);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> ErrorContext.instance().reset();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>具体是调用了 <code>Configuration</code> 的这个方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Executor <span class="title function_">newExecutor</span><span class="params">(Transaction transaction, ExecutorType executorType)</span> {</span><br><span class="line"> executorType = executorType == <span class="literal">null</span> ? defaultExecutorType : executorType;</span><br><span class="line"> Executor executor;</span><br><span class="line"> <span class="keyword">if</span> (ExecutorType.BATCH == executorType) {</span><br><span class="line"> executor = <span class="keyword">new</span> <span class="title class_">BatchExecutor</span>(<span class="built_in">this</span>, transaction);</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (ExecutorType.REUSE == executorType) {</span><br><span class="line"> executor = <span class="keyword">new</span> <span class="title class_">ReuseExecutor</span>(<span class="built_in">this</span>, transaction);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// ---------> 会走到这个分支</span></span><br><span class="line"> executor = <span class="keyword">new</span> <span class="title class_">SimpleExecutor</span>(<span class="built_in">this</span>, transaction);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (cacheEnabled) {</span><br><span class="line"> executor = <span class="keyword">new</span> <span class="title class_">CachingExecutor</span>(executor);</span><br><span class="line"> }</span><br><span class="line"> executor = (Executor) interceptorChain.pluginAll(executor);</span><br><span class="line"> <span class="keyword">return</span> executor;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>上面传入的 <code>executorType</code> 是 <code>Configuration</code> 的默认类型，也就是 <code>simple</code> 类型，并且 <code>cacheEnabled</code> 在 <code>Configuration</code> 默认为 <code>true</code>，所以会包装成<code>CachingExecutor</code> ,然后后面就是插件了，这块我们先不展开<br>然后我们的openSession返回的就是创建了<code>DefaultSqlSession</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">DefaultSqlSession</span><span class="params">(Configuration configuration, Executor executor, <span class="type">boolean</span> autoCommit)</span> {</span><br><span class="line"> <span class="built_in">this</span>.configuration = configuration;</span><br><span class="line"> <span class="built_in">this</span>.executor = executor;</span><br><span class="line"> <span class="built_in">this</span>.dirty = <span class="literal">false</span>;</span><br><span class="line"> <span class="built_in">this</span>.autoCommit = autoCommit;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后就是调用 <code>selectOne</code>, 因为前面已经把这部分代码说过了，就直接跳转过来<br><code>org.apache.ibatis.session.defaults.DefaultSqlSession#selectList(java.lang.String, java.lang.Object, org.apache.ibatis.session.RowBounds, org.apache.ibatis.session.ResultHandler)</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <E> List<E> <span class="title function_">selectList</span><span class="params">(String statement, Object parameter, RowBounds rowBounds, ResultHandler handler)</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">MappedStatement</span> <span class="variable">ms</span> <span class="operator">=</span> configuration.getMappedStatement(statement);</span><br><span class="line"> <span class="keyword">return</span> executor.query(ms, wrapCollection(parameter), rowBounds, handler);</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> ExceptionFactory.wrapException(<span class="string">"Error querying database. Cause: "</span> + e, e);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> ErrorContext.instance().reset();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>因为前面说了 <code>executor</code> 包装了 <code>CachingExecutor</code> ，所以会先调用</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <E> List<E> <span class="title function_">query</span><span class="params">(MappedStatement ms, Object parameterObject, RowBounds rowBounds, ResultHandler resultHandler)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="type">BoundSql</span> <span class="variable">boundSql</span> <span class="operator">=</span> ms.getBoundSql(parameterObject);</span><br><span class="line"> <span class="type">CacheKey</span> <span class="variable">key</span> <span class="operator">=</span> createCacheKey(ms, parameterObject, rowBounds, boundSql);</span><br><span class="line"> <span class="keyword">return</span> query(ms, parameterObject, rowBounds, resultHandler, key, boundSql);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后是调用的真实的query方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <E> List<E> <span class="title function_">query</span><span class="params">(MappedStatement ms, Object parameterObject, RowBounds rowBounds, ResultHandler resultHandler, CacheKey key, BoundSql boundSql)</span></span><br><span class="line"> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="type">Cache</span> <span class="variable">cache</span> <span class="operator">=</span> ms.getCache();</span><br><span class="line"> <span class="keyword">if</span> (cache != <span class="literal">null</span>) {</span><br><span class="line"> flushCacheIfRequired(ms);</span><br><span class="line"> <span class="keyword">if</span> (ms.isUseCache() && resultHandler == <span class="literal">null</span>) {</span><br><span class="line"> ensureNoOutParams(ms, boundSql);</span><br><span class="line"> <span class="meta">@SuppressWarnings("unchecked")</span></span><br><span class="line"> List<E> list = (List<E>) tcm.getObject(cache, key);</span><br><span class="line"> <span class="keyword">if</span> (list == <span class="literal">null</span>) {</span><br><span class="line"> list = delegate.query(ms, parameterObject, rowBounds, resultHandler, key, boundSql);</span><br><span class="line"> tcm.putObject(cache, key, list); <span class="comment">// issue #578 and #116</span></span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> list;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> delegate.query(ms, parameterObject, rowBounds, resultHandler, key, boundSql);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这里是第一次查询，没有缓存就先到最后一行，继续是调用到 <code>org.apache.ibatis.executor.BaseExecutor#queryFromDatabase</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <E> List<E> <span class="title function_">query</span><span class="params">(MappedStatement ms, Object parameter, RowBounds rowBounds, ResultHandler resultHandler, CacheKey key, BoundSql boundSql)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> ErrorContext.instance().resource(ms.getResource()).activity(<span class="string">"executing a query"</span>).object(ms.getId());</span><br><span class="line"> <span class="keyword">if</span> (closed) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">ExecutorException</span>(<span class="string">"Executor was closed."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (queryStack == <span class="number">0</span> && ms.isFlushCacheRequired()) {</span><br><span class="line"> clearLocalCache();</span><br><span class="line"> }</span><br><span class="line"> List<E> list;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> queryStack++;</span><br><span class="line"> list = resultHandler == <span class="literal">null</span> ? (List<E>) localCache.getObject(key) : <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">if</span> (list != <span class="literal">null</span>) {</span><br><span class="line"> handleLocallyCachedOutputParameters(ms, key, parameter, boundSql);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// ----------->会走到这里</span></span><br><span class="line"> list = queryFromDatabase(ms, parameter, rowBounds, resultHandler, key, boundSql);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> queryStack--;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (queryStack == <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">for</span> (DeferredLoad deferredLoad : deferredLoads) {</span><br><span class="line"> deferredLoad.load();</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// issue #601</span></span><br><span class="line"> deferredLoads.clear();</span><br><span class="line"> <span class="keyword">if</span> (configuration.getLocalCacheScope() == LocalCacheScope.STATEMENT) {</span><br><span class="line"> <span class="comment">// issue #482</span></span><br><span class="line"> clearLocalCache();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> list;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后是</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <E> List<E> <span class="title function_">queryFromDatabase</span><span class="params">(MappedStatement ms, Object parameter, RowBounds rowBounds, ResultHandler resultHandler, CacheKey key, BoundSql boundSql)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> List<E> list;</span><br><span class="line"> localCache.putObject(key, EXECUTION_PLACEHOLDER);</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> list = doQuery(ms, parameter, rowBounds, resultHandler, boundSql);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> localCache.removeObject(key);</span><br><span class="line"> }</span><br><span class="line"> localCache.putObject(key, list);</span><br><span class="line"> <span class="keyword">if</span> (ms.getStatementType() == StatementType.CALLABLE) {</span><br><span class="line"> localOutputParameterCache.putObject(key, parameter);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> list;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后就是 <code>simpleExecutor</code> 的执行过程</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <E> List<E> <span class="title function_">doQuery</span><span class="params">(MappedStatement ms, Object parameter, RowBounds rowBounds, ResultHandler resultHandler, BoundSql boundSql)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="type">Statement</span> <span class="variable">stmt</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">Configuration</span> <span class="variable">configuration</span> <span class="operator">=</span> ms.getConfiguration();</span><br><span class="line"> <span class="type">StatementHandler</span> <span class="variable">handler</span> <span class="operator">=</span> configuration.newStatementHandler(wrapper, ms, parameter, rowBounds, resultHandler, boundSql);</span><br><span class="line"> stmt = prepareStatement(handler, ms.getStatementLog());</span><br><span class="line"> <span class="keyword">return</span> handler.query(stmt, resultHandler);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> closeStatement(stmt);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>接下去其实就是跟jdbc交互了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <E> List<E> <span class="title function_">query</span><span class="params">(Statement statement, ResultHandler resultHandler)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="type">PreparedStatement</span> <span class="variable">ps</span> <span class="operator">=</span> (PreparedStatement) statement;</span><br><span class="line"> ps.execute();</span><br><span class="line"> <span class="keyword">return</span> resultSetHandler.handleResultSets(ps);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p><code>com.mysql.cj.jdbc.ClientPreparedStatement#execute</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">execute</span><span class="params">()</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">synchronized</span>(<span class="built_in">this</span>.checkClosed().getConnectionMutex()) {</span><br><span class="line"> <span class="type">JdbcConnection</span> <span class="variable">locallyScopedConn</span> <span class="operator">=</span> <span class="built_in">this</span>.connection;</span><br><span class="line"> <span class="keyword">if</span> (!<span class="built_in">this</span>.doPingInstead && !<span class="built_in">this</span>.checkReadOnlySafeStatement()) {</span><br><span class="line"> <span class="keyword">throw</span> SQLError.createSQLException(Messages.getString(<span class="string">"PreparedStatement.20"</span>) + Messages.getString(<span class="string">"PreparedStatement.21"</span>), <span class="string">"S1009"</span>, <span class="built_in">this</span>.exceptionInterceptor);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="type">ResultSetInternalMethods</span> <span class="variable">rs</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="built_in">this</span>.lastQueryIsOnDupKeyUpdate = <span class="literal">false</span>;</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.retrieveGeneratedKeys) {</span><br><span class="line"> <span class="built_in">this</span>.lastQueryIsOnDupKeyUpdate = <span class="built_in">this</span>.containsOnDuplicateKeyUpdate();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.batchedGeneratedKeys = <span class="literal">null</span>;</span><br><span class="line"> <span class="built_in">this</span>.resetCancelledState();</span><br><span class="line"> <span class="built_in">this</span>.implicitlyCloseAllOpenResults();</span><br><span class="line"> <span class="built_in">this</span>.clearWarnings();</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.doPingInstead) {</span><br><span class="line"> <span class="built_in">this</span>.doPingInstead();</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="built_in">this</span>.setupStreamingTimeout(locallyScopedConn);</span><br><span class="line"> <span class="type">Message</span> <span class="variable">sendPacket</span> <span class="operator">=</span> ((PreparedQuery)<span class="built_in">this</span>.query).fillSendPacket(((PreparedQuery)<span class="built_in">this</span>.query).getQueryBindings());</span><br><span class="line"> <span class="type">String</span> <span class="variable">oldDb</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">if</span> (!locallyScopedConn.getDatabase().equals(<span class="built_in">this</span>.getCurrentDatabase())) {</span><br><span class="line"> oldDb = locallyScopedConn.getDatabase();</span><br><span class="line"> locallyScopedConn.setDatabase(<span class="built_in">this</span>.getCurrentDatabase());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">CachedResultSetMetaData</span> <span class="variable">cachedMetadata</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">cacheResultSetMetadata</span> <span class="operator">=</span> (Boolean)locallyScopedConn.getPropertySet().getBooleanProperty(PropertyKey.cacheResultSetMetadata).getValue();</span><br><span class="line"> <span class="keyword">if</span> (cacheResultSetMetadata) {</span><br><span class="line"> cachedMetadata = locallyScopedConn.getCachedMetaData(((PreparedQuery)<span class="built_in">this</span>.query).getOriginalSql());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> locallyScopedConn.setSessionMaxRows(<span class="built_in">this</span>.getQueryInfo().getFirstStmtChar() == <span class="string">'S'</span> ? <span class="built_in">this</span>.maxRows : -<span class="number">1</span>);</span><br><span class="line"> rs = <span class="built_in">this</span>.executeInternal(<span class="built_in">this</span>.maxRows, sendPacket, <span class="built_in">this</span>.createStreamingResultSet(), <span class="built_in">this</span>.getQueryInfo().getFirstStmtChar() == <span class="string">'S'</span>, cachedMetadata, <span class="literal">false</span>);</span><br><span class="line"> <span class="keyword">if</span> (cachedMetadata != <span class="literal">null</span>) {</span><br><span class="line"> locallyScopedConn.initializeResultsMetadataFromCache(((PreparedQuery)<span class="built_in">this</span>.query).getOriginalSql(), cachedMetadata, rs);</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (rs.hasRows() && cacheResultSetMetadata) {</span><br><span class="line"> locallyScopedConn.initializeResultsMetadataFromCache(((PreparedQuery)<span class="built_in">this</span>.query).getOriginalSql(), (CachedResultSetMetaData)<span class="literal">null</span>, rs);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.retrieveGeneratedKeys) {</span><br><span class="line"> rs.setFirstCharOfQuery(<span class="built_in">this</span>.getQueryInfo().getFirstStmtChar());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (oldDb != <span class="literal">null</span>) {</span><br><span class="line"> locallyScopedConn.setDatabase(oldDb);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (rs != <span class="literal">null</span>) {</span><br><span class="line"> <span class="built_in">this</span>.lastInsertId = rs.getUpdateID();</span><br><span class="line"> <span class="built_in">this</span>.results = rs;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> rs != <span class="literal">null</span> && rs.hasRows();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (CJException var11) {</span><br><span class="line"> <span class="keyword">throw</span> SQLExceptionsMapping.translateException(var11, <span class="built_in">this</span>.getExceptionInterceptor());</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Mybatis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Mysql</tag>
|
|
<tag>Mybatis</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>nginx 日志小记</title>
|
|
<url>/2022/04/17/nginx-%E6%97%A5%E5%BF%97%E5%B0%8F%E8%AE%B0/</url>
|
|
<content><![CDATA[<p>nginx 默认的日志有特定的格式，我们也可以自定义，</p>
|
|
<p>默认的格式是预定义的 combined</p>
|
|
<figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">log_format combined <span class="string">'$remote_addr - $remote_user [$time_local] '</span></span><br><span class="line"> <span class="string">'"$request" $status $body_bytes_sent '</span></span><br><span class="line"> <span class="string">'"$http_referer" "$http_user_agent"'</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>配置的日志可以使用这个默认的，如果满足需求的话</p>
|
|
<figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">Syntax: access_log path [format [buffer=size] [gzip[=level]] [flush=time] [<span class="keyword">if</span>=condition]];</span><br><span class="line"> access_log off;</span><br><span class="line">Default: access_log logs/access.log combined;</span><br><span class="line">Context: http, server, location, <span class="keyword">if</span> <span class="keyword">in</span> location, limit_except</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>而如果需要额外的一些配置的话可以自己定义 <code>log_format</code> ，比如我想要给日志里加上请求时间，那就可以自己定义一个 <code>log_forma</code>t 比如添加下</p>
|
|
<figure class="highlight bash"><table><tr><td class="code"><pre><span class="line"><span class="variable">$request_time</span></span><br><span class="line">request processing time <span class="keyword">in</span> seconds with a milliseconds resolution; </span><br><span class="line">time elapsed between the first bytes were <span class="built_in">read</span> from the client and the <span class="built_in">log</span> write after the last bytes were sent to the client</span><br></pre></td></tr></table></figure>
|
|
|
|
<figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">log_format combined_extend <span class="string">'$remote_addr - $remote_user [$time_local] '</span></span><br><span class="line"> <span class="string">'"$request" $status $body_bytes_sent '</span></span><br><span class="line"> <span class="string">'"$http_referer" "$http_user_agent" "$request_time"'</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后其他的比如还有 gzip 压缩，可以设置压缩级别，flush 刷盘时间还有根据条件控制</p>
|
|
<p>这里的条件控制简单看了下还比较厉害</p>
|
|
<p>比如我想对2xx 跟 3xx 的访问不记录日志</p>
|
|
<figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">map <span class="variable">$status</span> <span class="variable">$loggable</span> {</span><br><span class="line"> ~^[23] 0;</span><br><span class="line"> default 1;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">access_log /path/to/access.log combined <span class="keyword">if</span>=<span class="variable">$loggable</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>当 <code>$loggable</code> 是 0 或者空时表示 if 条件为否，上面的默认就是 1，只有当请求状态 status 是 2xx 或 3xx 时才是 0，代表不用记录，有了这个特性就可以更灵活地配置日志</p>
|
|
<p>文章主要参考了 nginx 的 log 模块的<a href="%5Bhttp://nginx.org/en/docs/http/ngx_http_log_module.html%5D(http://nginx.org/en/docs/http/ngx_http_log_module.html)">文档</a></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>nginx</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>nginx</tag>
|
|
<tag>日志</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>java的字节码工具-javassist初体验</title>
|
|
<url>/2025/01/05/java%E7%9A%84%E5%AD%97%E8%8A%82%E7%A0%81%E5%B7%A5%E5%85%B7-javassist%E5%88%9D%E4%BD%93%E9%AA%8C/</url>
|
|
<content><![CDATA[<p>前面那篇在讲agent的时候用到了javassist，我们就来简单讲个demo<br>我想用javassist来创建一个类</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">createEntity</span><span class="params">()</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="type">ClassPool</span> <span class="variable">pool</span> <span class="operator">=</span> ClassPool.getDefault();</span><br><span class="line"> <span class="type">CtClass</span> <span class="variable">cl</span> <span class="operator">=</span> pool.makeClass(<span class="string">"Entity"</span>);</span><br><span class="line"> <span class="type">CtConstructor</span> <span class="variable">cons</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">CtConstructor</span>(<span class="keyword">new</span> <span class="title class_">CtClass</span>[]{}, cl);</span><br><span class="line"> cl.addConstructor(cons);</span><br><span class="line"> cl.writeFile();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这个代码可以给我们创建一个Entity类, 带有一个无参的构造函数，默认在项目根目录下会生成Entity类的class文件<br>但是其实这个也是不必要的，默认会给生成一个无参的构造函数<br>然后我们可以给这个类加个字段, 加在 writeFile 方法调用前</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">CtField</span> <span class="variable">param</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">CtField</span>(pool.get(<span class="string">"java.lang.String"</span>), <span class="string">"name"</span>, cl);</span><br><span class="line">param.setModifiers(Modifier.PRIVATE);</span><br><span class="line">cl.addField(param, CtField.Initializer.constant(<span class="string">"ent1"</span>));</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/blog/5nVlZU.png"><br>我们就生成了这样一个类<br>javassist也已经提供好了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">cl.addMethod(CtNewMethod.setter(<span class="string">"setName"</span>, param));</span><br><span class="line">cl.addMethod(CtNewMethod.getter(<span class="string">"getName"</span>, param));</span><br></pre></td></tr></table></figure>
|
|
<p>我们生成的类就变成这样了<br><img data-src="https://img.nicksxs.me/blog/UNLO6Y.png"><br>接下去我们可以再加一个默认赋值的无参构造函数</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">CtConstructor</span> <span class="variable">con</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">CtConstructor</span>(<span class="keyword">new</span> <span class="title class_">CtClass</span>[]{}, cl);</span><br><span class="line"><span class="comment">// 需要转义下</span></span><br><span class="line">con.setBody(<span class="string">"{name = \"nick\";}"</span>);</span><br><span class="line">cl.addConstructor(con);</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/blog/s36dQa.png"><br>然后我们就有了这样的类，带有一个默认赋值的构造函数，接下去可以来个带参数的构造函数</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">con = <span class="keyword">new</span> <span class="title class_">CtConstructor</span>(<span class="keyword">new</span> <span class="title class_">CtClass</span>[]{pool.get(<span class="string">"java.lang.String"</span>)}, cl);</span><br><span class="line"><span class="comment">// $0=this / $1,$2,$3...指的是参数，$1代表第一个参数</span></span><br><span class="line">con.setBody(<span class="string">"{$0.name = $1;}"</span>);</span><br><span class="line">cl.addConstructor(con);</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/blog/GEuefW.png"><br>最后我们可以再加个打印该字段的方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">CtMethod</span> <span class="variable">ctMethod</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">CtMethod</span>(CtClass.voidType, <span class="string">"printName"</span>, <span class="keyword">new</span> <span class="title class_">CtClass</span>[]{}, cl);</span><br><span class="line">ctMethod.setModifiers(Modifier.PUBLIC);</span><br><span class="line">ctMethod.setBody(<span class="string">"{System.out.println(name);}"</span>);</span><br><span class="line">cl.addMethod(ctMethod);</span><br></pre></td></tr></table></figure>
|
|
<p>这样我们就获得了这样一个类<br><img data-src="https://img.nicksxs.me/blog/u0RVvD.png"><br>这只是javassist的简单使用，它还有很多强大的功能可以在<a href="https://www.javassist.org/tutorial/tutorial.html">官方文档</a>和网上的资料进行学习</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>openresty</title>
|
|
<url>/2019/06/18/openresty/</url>
|
|
<content><![CDATA[<p>目前公司要对一些新的产品功能做灰度测试，因为在后端业务代码层面添加判断比较麻烦，所以想在nginx上做点手脚，就想到了openresty<br>前后也踩了不少坑，这边先写一点</p>
|
|
<p>首先是日志<br><code>error_log logs/error.log debug;</code><br>需要nginx开启日志的debug才能看到日志</p>
|
|
<p>使用 <code>lua_code_cache off</code>即可, 另外注意只有使用 <code>content_by_lua_file</code> 才会生效</p>
|
|
<figure class="highlight lua"><table><tr><td class="code"><pre><span class="line">http {</span><br><span class="line"> lua_code_cache off;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">location ~* /(\d+-.*)/api/orgunits/load_all(.*) {</span><br><span class="line"> default_type <span class="string">'application/json;charset=utf-8'</span>;</span><br><span class="line"> content_by_lua_file /data/projects/xxx/current/lua/controller/load_data.lua;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>使用lua给nginx请求response头添加内容可以用这个</p>
|
|
<figure class="highlight lua"><table><tr><td class="code"><pre><span class="line">ngx.header[<span class="string">'response'</span>] = <span class="string">'header'</span></span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
<p><a href="http://cyukang.com/2017/05/22/try-on-openresty.html">使用总结</a></p>
|
|
<p>后续：</p>
|
|
<ol>
|
|
<li><p>一开始在本地环境的时候使用content_by_lua_file只关注了头，后来发到测试环境发现请求内容都没代理转发到后端服务上<br>网上查了下发现content_by_lua_file是将请求的所有内容包括response都用这里面的lua脚本生成了，content这个词就表示是请求内容<br>后来改成了access_by_lua_file就正常了，只是要去获取请求内容和修改响应头，并不是要完整的接管请求</p>
|
|
</li>
|
|
<li><p>后来又碰到了一个坑是nginx有个client_body_buffer_size的配置参数，nginx在32位和64位系统里有8K和16K两个默认值，当请求内容大于这两个值的时候，会把请求内容放到临时文件里，这个时候openresty里的ngx.req.get_post_args()就会报“failed to get post args: requesty body in temp file not supported”这个错误，将client_body_buffer_size这个参数配置调大一点就好了</p>
|
|
</li>
|
|
<li><p>还有就是lua的异常捕获，网上看一般是用pcall和xpcall来进行保护调用，因为问题主要出在cjson的decode，这里有两个解决方案，一个就是将cjson.decode使用pcall封装，</p>
|
|
<figure class="highlight lua"><table><tr><td class="code"><pre><span class="line"><span class="keyword">local</span> decode = <span class="built_in">require</span>(<span class="string">"cjson"</span>).decode</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">function</span> <span class="title">json_decode</span><span class="params">( str )</span></span></span><br><span class="line"> <span class="keyword">local</span> ok, t = <span class="built_in">pcall</span>(decode, str)</span><br><span class="line"> <span class="keyword">if</span> <span class="keyword">not</span> ok <span class="keyword">then</span></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">nil</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> t</span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure>
|
|
<p> 这个是使用了pcall，称为保护调用，会在内部错误后返回两个参数，第一个是false，第二个是错误信息<br> 还有一种是使用cjson.safe包</p>
|
|
<figure class="highlight lua"><table><tr><td class="code"><pre><span class="line"><span class="keyword">local</span> json = <span class="built_in">require</span>(<span class="string">"cjson.safe"</span>)</span><br><span class="line"><span class="keyword">local</span> str = <span class="string">[[ {"key:"value"} ]]</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">local</span> t = json.decode(str)</span><br><span class="line"><span class="keyword">if</span> t <span class="keyword">then</span></span><br><span class="line"> ngx.say(<span class="string">" --> "</span>, <span class="built_in">type</span>(t))</span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure>
|
|
<p> cjson.safe包会在解析失败的时候返回nil</p>
|
|
</li>
|
|
<li><p>还有一个是redis链接时如果host使用的是域名的话会提示“failed to connect: no resolver defined to resolve “redis.xxxxxx.com””，这里需要使用nginx的resolver指令，<br> <code>resolver 8.8.8.8 valid=3600s;</code></p>
|
|
</li>
|
|
<li><p>还有一点补充下<br> 就是业务在使用redis的时候使用了db的特性，所以在lua访问redis的时候也需要执行db，这里lua的redis库也支持了这个特性，可以使用instance:select(config:get(‘db’))来切换db</p>
|
|
</li>
|
|
<li><p>性能优化tips<br><a href="https://juejin.im/entry/5b0e8fcef265da09210072a3#openresty">建议是尽量少使用阶段钩子，例如content_by_lua_file，*_by_lua</a></p>
|
|
</li>
|
|
<li><p>发现一个不错的openresty站点<br><a href="http://orhub.org/">地址</a></p>
|
|
</li>
|
|
</ol>
|
|
]]></content>
|
|
<categories>
|
|
<category>nginx</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>nginx</tag>
|
|
<tag>openresty</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>pcre-intro-and-a-simple-package</title>
|
|
<url>/2015/01/16/pcre-intro-and-a-simple-package/</url>
|
|
<content><![CDATA[<h3 id="Pcre"><a href="#Pcre" class="headerlink" title="Pcre"></a><a href="http://www.pcre.org/">Pcre</a></h3><blockquote>
|
|
<p>Perl Compatible Regular Expressions (PCRE) is a regular<br> expression C library inspired by the regular expression<br> capabilities in the Perl programming language, written<br> by Philip Hazel, starting in summer 1997.</p>
|
|
</blockquote>
|
|
<p>因为最近工作内容的一部分需要做字符串的识别处理，所以就顺便用上了之前在PHP中用过的正则，在C/C++中本身不包含正则库，这里使用的pcre，对MFC开发，在<a href="http://www.psyon.org/projects/pcre-win32/index.php">这里</a>提供了静态链接库，在引入lib跟.h文件后即可使用。</p>
|
|
<span id="more"></span>
|
|
|
|
<h3 id="Regular-Expression-Syntax"><a href="#Regular-Expression-Syntax" class="headerlink" title="Regular Expression Syntax"></a>Regular Expression Syntax</h3><p>然后是一些<a href="http://www.pcre.org/original/doc/html/pcresyntax.html">正则语法</a>，官方的语法文档比较科学严谨，特别是对类似于贪婪匹配等细节的说明，当然一般的使用可以在网上找到很多匹配语法，例如<a href="http://www.regextester.com/pregsyntax.html">这个</a>。</p>
|
|
<h3 id="PCRE函数介绍"><a href="#PCRE函数介绍" class="headerlink" title="PCRE函数介绍"></a>PCRE函数介绍</h3><blockquote>
|
|
<p>pcre_compile<br>原型：</p>
|
|
</blockquote>
|
|
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><pcre.h></span></span></span><br><span class="line"><span class="function">pcre *<span class="title">pcre_compile</span><span class="params">(<span class="type">const</span> <span class="type">char</span> *pattern, <span class="type">int</span> options, <span class="type">const</span> <span class="type">char</span> **errptr, <span class="type">int</span> *erroffset, <span class="type">const</span> <span class="type">unsigned</span> <span class="type">char</span> *tableptr)</span></span>;</span><br></pre></td></tr></table></figure>
|
|
<p>功能：将一个正则表达式编译成一个内部表示，在匹配多个字符串时，可以加速匹配。其同pcre_compile2功能一样只是缺少一个参数errorcodeptr。<br>参数：<br><code>pattern</code> 正则表达式<br><code>options</code> 为0，或者其他参数选项<br><code>errptr</code> 出错消息<br><code>erroffset</code> 出错位置<br><code>tableptr</code> 指向一个字符数组的指针，可以设置为空NULL</p>
|
|
<blockquote>
|
|
<p>pcre_exec<br>原型：</p>
|
|
</blockquote>
|
|
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><pcre.h></span></span></span><br><span class="line"><span class="function"><span class="type">int</span> <span class="title">pcre_exec</span><span class="params">(<span class="type">const</span> pcre *code, <span class="type">const</span> pcre_extra *extra, <span class="type">const</span> <span class="type">char</span> *subject, <span class="type">int</span> length, <span class="type">int</span> startoffset, <span class="type">int</span> options, <span class="type">int</span> *ovector, <span class="type">int</span> ovecsize)</span></span></span><br></pre></td></tr></table></figure>
|
|
<p>功能：使用编译好的模式进行匹配，采用与Perl相似的算法，返回匹配串的偏移位置。<br>参数：<br><code>code</code> 编译好的模式<br><code>extra</code> 指向一个pcre_extra结构体，可以为NULL<br><code>subject</code> 需要匹配的字符串<br><code>length</code> 匹配的字符串长度（Byte）<br><code>startoffset</code> 匹配的开始位置<br><code>options</code> 选项位<br><code>ovector</code> 指向一个结果的整型数组<br><code>ovecsize</code> 数组大小。</p>
|
|
<p>这里是两个最常用的函数的简单说明，pcre的静态库提供了一系列的函数以供使用，可以参考这个<a href="http://blog.csdn.net/sulliy/article/details/6247155">博客</a>说明，另外对于以上函数的具体参数详细说明可以参考官网<a href="http://www.pcre.org/original/doc/html/">此处</a></p>
|
|
<h3 id="一个丑陋的封装"><a href="#一个丑陋的封装" class="headerlink" title="一个丑陋的封装"></a>一个丑陋的封装</h3><figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="type">void</span> <span class="title">COcxDemoDlg::pcre_exec_all</span><span class="params">(<span class="type">const</span> pcre * re, PCRE_SPTR src, vector<pair<<span class="type">int</span>, <span class="type">int</span>>> &vc)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> <span class="type">int</span> rc;</span><br><span class="line"> <span class="type">int</span> ovector[<span class="number">30</span>];</span><br><span class="line"> <span class="type">int</span> i = <span class="number">0</span>;</span><br><span class="line"> pair<<span class="type">int</span>, <span class="type">int</span>> pr;</span><br><span class="line"> rc = <span class="built_in">pcre_exec</span>(re, <span class="literal">NULL</span>, src, <span class="built_in">strlen</span>(src), i, <span class="number">0</span>, ovector, <span class="number">30</span>);</span><br><span class="line"> <span class="keyword">for</span> (; rc > <span class="number">0</span>;)</span><br><span class="line"> {</span><br><span class="line"> i = ovector[<span class="number">1</span>];</span><br><span class="line"> pr.first = ovector[<span class="number">2</span>];</span><br><span class="line"> pr.second = ovector[<span class="number">3</span>];</span><br><span class="line"> vc.<span class="built_in">push_back</span>(pr);</span><br><span class="line"> rc = <span class="built_in">pcre_exec</span>(re, <span class="literal">NULL</span>, src, <span class="built_in">strlen</span>(src), i, <span class="number">0</span>, ovector, <span class="number">30</span>);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>vector中是全文匹配后的索引对，只是简单地用下。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>C++</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>c++</tag>
|
|
<tag>mfc</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>php-abstract-class-and-interface</title>
|
|
<url>/2016/11/10/php-abstract-class-and-interface/</url>
|
|
<content><![CDATA[<h2 id="PHP抽象类和接口"><a href="#PHP抽象类和接口" class="headerlink" title="PHP抽象类和接口"></a>PHP抽象类和接口</h2><ul>
|
|
<li>抽象类与接口</li>
|
|
<li>抽象类内可以包含非抽象函数，即可实现函数</li>
|
|
<li>抽象类内必须包含至少一个抽象方法，抽象类和接口均不能实例化</li>
|
|
<li>抽象类可以设置访问级别，接口默认都是public</li>
|
|
<li>类可以实现多个接口但不能继承多个抽象类</li>
|
|
<li>类必须实现抽象类和接口里的抽象方法，不一定要实现抽象类的非抽象方法</li>
|
|
<li>接口内不能定义变量，但是可以定义常量</li>
|
|
</ul>
|
|
<h2 id="示例代码"><a href="#示例代码" class="headerlink" title="示例代码"></a>示例代码</h2><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line"><?php</span><br><span class="line">interface int1{</span><br><span class="line"> const INTER1 = 111;</span><br><span class="line"> function inter1();</span><br><span class="line">}</span><br><span class="line">interface int2{</span><br><span class="line"> const INTER1 = 222;</span><br><span class="line"> function inter2();</span><br><span class="line">}</span><br><span class="line">abstract class abst1{</span><br><span class="line"> public function abstr1(){</span><br><span class="line"> echo 1111;</span><br><span class="line"> }</span><br><span class="line"> abstract function abstra1(){</span><br><span class="line"> echo 'ahahahha';</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line">abstract class abst2{</span><br><span class="line"> public function abstr2(){</span><br><span class="line"> echo 1111;</span><br><span class="line"> }</span><br><span class="line"> abstract function abstra2();</span><br><span class="line">}</span><br><span class="line">class normal1 extends abst1{</span><br><span class="line"> protected function abstr2(){</span><br><span class="line"> echo 222;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<h2 id="result"><a href="#result" class="headerlink" title="result"></a>result</h2><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">PHP Fatal error: Abstract function abst1::abstra1() cannot contain body in new.php on line 17</span><br><span class="line"></span><br><span class="line">Fatal error: Abstract function abst1::abstra1() cannot contain body in php on line 17</span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>php</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>php</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>php 的调试方法-查看调用堆栈</title>
|
|
<url>/2023/12/31/php-%E7%9A%84%E8%B0%83%E8%AF%95%E6%96%B9%E6%B3%95-%E6%9F%A5%E7%9C%8B%E8%B0%83%E7%94%A8%E5%A0%86%E6%A0%88/</url>
|
|
<content><![CDATA[<p>php 代码调试里的神器就是 <code>echo 111;exit;</code> 但是对于使用了接口和继承比较多的话，有时候比较难找，可能定位到了一段代码但是不知道怎么调用过来的，这时候就可以用这个方法</p>
|
|
<p>总结下来有三种，</p>
|
|
<h2 id="第一种是最巧妙的"><a href="#第一种是最巧妙的" class="headerlink" title="第一种是最巧妙的"></a>第一种是最巧妙的</h2><figure class="highlight php"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">function</span> <span class="title">a</span>(<span class="params"></span>) </span>{</span><br><span class="line"><span class="keyword">echo</span> <span class="number">111</span>;<span class="keyword">exit</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>比如本来是上面这样子，那么其实我们可以主动new 个异常</p>
|
|
<figure class="highlight php"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">function</span> <span class="title">a</span>(<span class="params"></span>) </span>{</span><br><span class="line"><span class="variable">$e</span> = <span class="keyword">new</span> <span class="built_in">Exception</span>();</span><br><span class="line"><span class="title function_ invoke__">print_r</span>(<span class="variable">$e</span>-><span class="title function_ invoke__">getTraceAsString</span>());</span><br><span class="line"><span class="keyword">echo</span> <span class="number">111</span>;<span class="keyword">exit</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这样我的 trace 调用链路就出来了</p>
|
|
<h2 id="第二种"><a href="#第二种" class="headerlink" title="第二种"></a>第二种</h2><p>这个就是比较简单的，调用 php 自身提供的方法</p>
|
|
<figure class="highlight php"><table><tr><td class="code"><pre><span class="line"><span class="title function_ invoke__">debug_backtrace</span>(<span class="keyword">int</span> <span class="variable">$options</span> = DEBUG_BACKTRACE_PROVIDE_OBJECT, <span class="keyword">int</span> <span class="variable">$limit</span> = <span class="number">0</span>): <span class="keyword">array</span></span><br></pre></td></tr></table></figure>
|
|
<p>第一个参数是个掩码</p>
|
|
<table>
|
|
<thead>
|
|
<tr>
|
|
<th>debug_backtrace()</th>
|
|
<th>Populates both indexes</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr>
|
|
<td>debug_backtrace(DEBUG_BACKTRACE_PROVIDE_OBJECT)</td>
|
|
<td></td>
|
|
</tr>
|
|
<tr>
|
|
<td>debug_backtrace(1)</td>
|
|
<td></td>
|
|
</tr>
|
|
<tr>
|
|
<td>debug_backtrace(0)</td>
|
|
<td>Omits index <code>"object"</code> and populates index <code>"args"</code>.</td>
|
|
</tr>
|
|
<tr>
|
|
<td>debug_backtrace(DEBUG_BACKTRACE_IGNORE_ARGS)</td>
|
|
<td>Omits index <code>"object"</code> <em>and</em> index <code>"args"</code>.</td>
|
|
</tr>
|
|
<tr>
|
|
<td>debug_backtrace(2)</td>
|
|
<td></td>
|
|
</tr>
|
|
<tr>
|
|
<td>debug_backtrace(DEBUG_BACKTRACE_PROVIDE_OBJECT</td>
|
|
<td>DEBUG_BACKTRACE_IGNORE_ARGS)</td>
|
|
</tr>
|
|
<tr>
|
|
<td>debug_backtrace(3)</td>
|
|
<td></td>
|
|
</tr>
|
|
</tbody></table>
|
|
<p>第二个参数是限制栈深度</p>
|
|
<h3 id="第三种"><a href="#第三种" class="headerlink" title="第三种"></a>第三种</h3><p>这个也是用自身的方法</p>
|
|
<figure class="highlight php"><table><tr><td class="code"><pre><span class="line"><span class="title function_ invoke__">debug_print_backtrace</span>(<span class="keyword">int</span> <span class="variable">$options</span> = <span class="number">0</span>, <span class="keyword">int</span> <span class="variable">$limit</span> = <span class="number">0</span>): <span class="keyword">void</span></span><br></pre></td></tr></table></figure>
|
|
<p>这里的第一个参数只有一个可以传的</p>
|
|
<table>
|
|
<thead>
|
|
<tr>
|
|
<th>DEBUG_BACKTRACE_IGNORE_ARGS</th>
|
|
<th>Whether or not to omit the “args” index, and thus all the function/method arguments, to save memory.</th>
|
|
</tr>
|
|
</thead>
|
|
</table>
|
|
<p>就是隐藏参数，不然如果对于一些框架代码，这个打印会非常大，需要注意下</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>php</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>php</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>powershell 初体验</title>
|
|
<url>/2022/11/13/powershell-%E5%88%9D%E4%BD%93%E9%AA%8C/</url>
|
|
<content><![CDATA[<h3 id="powershell变量"><a href="#powershell变量" class="headerlink" title="powershell变量"></a>powershell变量</h3><p>变量命名类似于php</p>
|
|
<figure class="highlight powershell"><table><tr><td class="code"><pre><span class="line"><span class="built_in">PS</span> C:\Users\Nicks> <span class="variable">$a</span>=<span class="number">1</span></span><br><span class="line"><span class="built_in">PS</span> C:\Users\Nicks> <span class="variable">$b</span>=<span class="number">2</span></span><br><span class="line"><span class="built_in">PS</span> C:\Users\Nicks> <span class="variable">$a</span>*<span class="variable">$b</span></span><br><span class="line"><span class="number">2</span></span><br></pre></td></tr></table></figure>
|
|
<p>有一个比较好用的是变量交换<br>一般的语言做两个变量交换一般需要一个临时变量</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_">$</span><span class="language-bash">tmp=<span class="variable">$a</span></span></span><br><span class="line"><span class="meta prompt_">$</span><span class="language-bash">a=<span class="variable">$b</span></span></span><br><span class="line"><span class="meta prompt_">$</span><span class="language-bash">b=<span class="variable">$tmp</span></span></span><br></pre></td></tr></table></figure>
|
|
<p>而在powershell中可以这样</p>
|
|
<figure class="highlight powershell"><table><tr><td class="code"><pre><span class="line"><span class="variable">$a</span>,<span class="variable">$b</span>=<span class="variable">$b</span>,<span class="variable">$a</span></span><br><span class="line"><span class="built_in">PS</span> C:\Users\Nicks> <span class="variable">$a</span>,<span class="variable">$b</span>=<span class="variable">$b</span>,<span class="variable">$a</span></span><br><span class="line"><span class="built_in">PS</span> C:\Users\Nicks> <span class="variable">$a</span></span><br><span class="line"><span class="number">2</span></span><br><span class="line"><span class="built_in">PS</span> C:\Users\Nicks> <span class="variable">$b</span></span><br><span class="line"><span class="number">1</span></span><br></pre></td></tr></table></figure>
|
|
<p>还可以通过这个</p>
|
|
<figure class="highlight powershell"><table><tr><td class="code"><pre><span class="line"><span class="built_in">PS</span> C:\Users\Nicks> <span class="built_in">ls</span> variable:</span><br><span class="line"></span><br><span class="line">Name Value</span><br><span class="line"><span class="literal">----</span> <span class="literal">-----</span></span><br><span class="line"><span class="variable">$</span> <span class="variable">$b</span></span><br><span class="line">? True</span><br><span class="line">^ <span class="variable">$b</span></span><br><span class="line">a <span class="number">2</span></span><br><span class="line">args {}</span><br><span class="line">b <span class="number">1</span></span><br></pre></td></tr></table></figure>
|
|
<p>查看现存的变量<br>当然一般脚本都是动态类型的，<br>可以通过<br>gettype方法<br><img data-src="https://img.nicksxs.me/blog/rGDt0n.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>语言</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>powershell</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>rabbitmq-tips</title>
|
|
<url>/2017/04/25/rabbitmq-tips/</url>
|
|
<content><![CDATA[<h2 id="rabbitmq-介绍"><a href="#rabbitmq-介绍" class="headerlink" title="rabbitmq 介绍"></a>rabbitmq 介绍</h2><p>接触了一下rabbitmq，原来在选型的时候是在rabbitmq跟kafka之间做选择，网上搜了一下之后发现kafka的优势在于吞吐量，而rabbitmq相对注重可靠性，因为应用在im上，需要保证消息不能丢失所以就暂时选定rabbitmq，<br>Message Queue的需求由来已久，80年代最早在金融交易中，高盛等公司采用Teknekron公司的产品，当时的Message queuing软件叫做：the information bus（TIB）。 TIB被电信和通讯公司采用，路透社收购了Teknekron公司。之后，IBM开发了MQSeries，微软开发了Microsoft Message Queue（MSMQ）。这些商业MQ供应商的问题是厂商锁定，价格高昂。2001年，Java Message queuing试图解决锁定和交互性的问题，但对应用来说反而更加麻烦了。<br>RabbitMQ采用Erlang语言开发。Erlang语言由Ericson设计，专门为开发concurrent和distribution系统的一种语言，在电信领域使用广泛。OTP（Open Telecom Platform）作为Erlang语言的一部分，包含了很多基于Erlang开发的中间件／库／工具，如mnesia／SASL，极大方便了Erlang应用的开发。OTP就类似于Python语言中众多的module，用户借助这些module可以很方便的开发应用。<br>于是2004年，摩根大通和iMatrix开始着手<a href="https://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol">Advanced Message Queuing Protocol （AMQP）</a>开放标准的开发。2006年，AMQP规范发布。2007年，Rabbit技术公司基于AMQP标准开发的RabbitMQ 1.0 发布。所有主要的编程语言均有与代理接口通讯的客户端库。</p>
|
|
<h2 id="简单的使用经验"><a href="#简单的使用经验" class="headerlink" title="简单的使用经验"></a>简单的使用经验</h2><h3 id="通俗的理解"><a href="#通俗的理解" class="headerlink" title="通俗的理解"></a>通俗的理解</h3><p>这里介绍下其中的一些概念，connection表示和队列服务器的连接，一般情况下是tcp连接， channel表示通道，可以在一个连接上建立多个通道，这里主要是节省了tcp连接握手的成本，exchange可以理解成一个路由器，将消息推送给对应的队列queue，其实是像一个订阅的模式。</p>
|
|
<h3 id="集群经验"><a href="#集群经验" class="headerlink" title="集群经验"></a>集群经验</h3><p><code>rabbitmqctl stop</code>这个是关闭rabbitmq，在搭建集群时候先关闭服务，然后使用<code>rabbitmq-server -detached</code>静默启动，这时候使用<code>rabbitmqctl cluster_status</code>查看集群状态，因为还没将节点加入集群，所以只能看到类似</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">Cluster status of node rabbit@rabbit1 ...</span><br><span class="line">[{nodes,[{disc,[rabbit@rabbit1,rabbit@rabbit2,rabbit@rabbit3]}]},</span><br><span class="line"> {running_nodes,[rabbit@rabbit2,rabbit@rabbit1]}]</span><br><span class="line">...done.</span><br></pre></td></tr></table></figure>
|
|
<p>然后就可以把当前节点加入集群，</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_">rabbit2$ </span><span class="language-bash">rabbitmqctl stop_app <span class="comment">#这个stop_app与stop的区别是前者停的是rabbitmq应用，保留erlang节点，</span></span></span><br><span class="line"> #后者是停止了rabbitmq和erlang节点</span><br><span class="line">Stopping node rabbit@rabbit2 ...done.</span><br><span class="line"><span class="meta prompt_">rabbit2$ </span><span class="language-bash">rabbitmqctl join_cluster rabbit@rabbit1 <span class="comment">#这里可以用--ram指定将当前节点作为内存节点加入集群</span></span></span><br><span class="line">Clustering node rabbit@rabbit2 with [rabbit@rabbit1] ...done.</span><br><span class="line"><span class="meta prompt_">rabbit2$ </span><span class="language-bash">rabbitmqctl start_app</span></span><br><span class="line">Starting node rabbit@rabbit2 ...done.</span><br></pre></td></tr></table></figure>
|
|
<p>其他可以参考<a href="http://www.rabbitmq.com/clustering.html">官方文档</a></p>
|
|
<h2 id="一些坑"><a href="#一些坑" class="headerlink" title="一些坑"></a>一些坑</h2><h3 id="消息丢失"><a href="#消息丢失" class="headerlink" title="消息丢失"></a>消息丢失</h3><p>这里碰到过一个坑，对于使用exchange来做消息路由的，会有一个情况，就是在routing_key没被订阅的时候，会将该条找不到路由对应的queue的消息丢掉<code>What happens if we break our contract and send a message with one or four words, like "orange" or "quick.orange.male.rabbit"? Well, these messages won't match any bindings and will be lost.</code><a href="http://www.rabbitmq.com/tutorials/tutorial-five-python.html">对应链接</a>,而当使用空的exchange时，会保留消息，当出现消费者的时候就可以将收到之前生产者所推送的消息<a href="http://www.rabbitmq.com/tutorials/tutorial-two-python.html">对应链接</a>,这里就是用了空的exchange。</p>
|
|
<h3 id="集群搭建"><a href="#集群搭建" class="headerlink" title="集群搭建"></a>集群搭建</h3><p>集群搭建的时候有个erlang vm生成的random cookie，这个是用来做集群之间认证的，相同的cookie才能连接，但是如果通过vim打开复制后在其他几点新建文件写入会多一个换行，导致集群建立是报错，所以这里最好使用scp等传输命令直接传输cookie文件，同时要注意下cookie的文件权限。<br>另外在集群搭建的时候如果更改过hostname，那么要把rabbitmq的数据库删除，否则启动后会马上挂掉</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>php</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>php</tag>
|
|
<tag>mq</tag>
|
|
<tag>im</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>powershell 初体验二</title>
|
|
<url>/2022/11/20/powershell-%E5%88%9D%E4%BD%93%E9%AA%8C%E4%BA%8C/</url>
|
|
<content><![CDATA[<p>powershell创建数组也很方便<br>可以这样</p>
|
|
<figure class="highlight powershell"><table><tr><td class="code"><pre><span class="line"><span class="variable">$nums</span>=<span class="number">2</span>,<span class="number">0</span>,<span class="number">1</span>,<span class="number">2</span></span><br></pre></td></tr></table></figure>
|
|
<p>顺便可以用下我们上次学到的<code>gettype()</code><br><img data-src="https://img.nicksxs.me/blog/ntNjZZ.png"></p>
|
|
<p>如果是想创建连续数字的数组还可以用这个方便的方法</p>
|
|
<figure class="highlight powershell"><table><tr><td class="code"><pre><span class="line"><span class="variable">$nums</span>=<span class="number">1</span>..<span class="number">5</span></span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/blog/0Em1Mb.png"><br>而且数组还可以存放各种类型的数据</p>
|
|
<figure class="highlight powershell"><table><tr><td class="code"><pre><span class="line"><span class="variable">$array</span>=<span class="number">1</span>,<span class="string">"哈哈"</span>,([<span class="type">System.Guid</span>]::NewGuid()),(<span class="built_in">get-date</span>)</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/blog/ixU6Hp.png"><br>还有判断类型可以用<code>-is</code><br><img data-src="https://img.nicksxs.me/blog/eeWIP9.png"><br>创建一个空数组</p>
|
|
<figure class="highlight powershell"><table><tr><td class="code"><pre><span class="line"><span class="variable">$array</span>=<span class="selector-tag">@</span>()</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/blog/Bp9UAF.png"><br>数组添加元素</p>
|
|
<figure class="highlight powershell"><table><tr><td class="code"><pre><span class="line"><span class="variable">$array</span>+=<span class="string">"a"</span></span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/blog/iaXGt5.png"><br>数组删除元素</p>
|
|
<figure class="highlight powershell"><table><tr><td class="code"><pre><span class="line"><span class="variable">$a</span>=<span class="number">1</span>..<span class="number">4</span></span><br><span class="line"><span class="variable">$a</span>=<span class="variable">$a</span>[<span class="number">0</span><span class="type">..1</span>]+<span class="variable">$a</span>[<span class="number">3</span>]</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/blog/oUKE8A.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>语言</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>powershell</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>redis 的 rdb 和 COW 介绍</title>
|
|
<url>/2021/08/15/redis-%E7%9A%84-rdb-%E5%92%8C-COW-%E4%BB%8B%E7%BB%8D/</url>
|
|
<content><![CDATA[<p>redis 在使用 rdb 策略进行备份时，rdb 的意思是会在开启备份的时候将开启时间点的内存数据进行备份，并且可以设置时间，这样子就是这个策略其实还是不完全可靠的，如果是在这个间隔中宕机了，或者间隔过长，不过这个不在这次的要说的内容中，如果自己去写这个 rdb 的策略可能就有点类似于 mvcc 的 redolog，需要知道这个时间点之前的数据是怎么样的，防止后面更改的干扰，但是这样一方面需要有比较复杂的 mvcc 实现，另一方面是很占用存储空间，所以 redis 在这里面使用了 COW (Copy On Write) 技术，这个技术呢以前听过，也大致了解是怎么个意思，这次稍微具体地来看下，其实 redis 的 copy-on-write 就是来自于 linux 的 cow</p>
|
|
<h3 id="Linux中的CopyOnWrite"><a href="#Linux中的CopyOnWrite" class="headerlink" title="Linux中的CopyOnWrite"></a>Linux中的CopyOnWrite</h3><p>fork()之后，kernel把父进程中所有的内存页的权限都设为read-only，然后子进程的地址空间指向父进程。当父子进程都只读内存时，相安无事。当其中某个进程写内存时，CPU硬件检测到内存页是read-only的，于是触发页异常中断（page-fault），陷入kernel的一个中断例程。中断例程中，kernel就会把触发的异常的页复制一份，于是父子进程各自持有独立的一份。这个操作其实可以类比为写屏障，正常的读取是没问题的，当有写入时就会分裂。</p>
|
|
<h3 id="CopyOnWrite的好处："><a href="#CopyOnWrite的好处：" class="headerlink" title="CopyOnWrite的好处："></a>CopyOnWrite的好处：</h3><p>1、减少分配和复制资源时带来的瞬时延迟；<br>2、减少不必要的资源分配；<br>CopyOnWrite的缺点：<br>1、如果父子进程都需要进行大量的写操作，会产生大量的分页错误（页异常中断page-fault）;</p>
|
|
<h3 id="Redis中的CopyOnWrite"><a href="#Redis中的CopyOnWrite" class="headerlink" title="Redis中的CopyOnWrite"></a>Redis中的CopyOnWrite</h3><p>Redis在持久化时，如果是采用BGSAVE命令或者BGREWRITEAOF的方式，那Redis会fork出一个子进程来读取数据，从而写到磁盘中。<br>总体来看，Redis还是读操作比较多。如果子进程存在期间，发生了大量的写操作，那可能就会出现很多的分页错误(页异常中断page-fault)，这样就得耗费不少性能在复制上。<br>而在rehash阶段上，写操作是无法避免的。所以Redis在fork出子进程之后，将负载因子阈值提高，尽量减少写操作，避免不必要的内存写入操作，最大限度地节约内存。这里其实更巧妙了，在细节上去优化会产生大量页异常中断的情况。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>redis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>redis</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>redis数据结构介绍-第一部分 SDS，链表，字典</title>
|
|
<url>/2019/12/26/redis%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84%E4%BB%8B%E7%BB%8D/</url>
|
|
<content><![CDATA[<p>redis是现在服务端很常用的缓存中间件，其实原来还有<code>memcache</code>之类的竞品，但是现在貌似 redis 快一统江湖，这里当然不是在吹，只是个人角度的一个感觉，不权威只是主观感觉。<br>redis 主要有五种数据结构，<code>Strings</code>，<code>Lists</code>，<code>Sets</code>，<code>Hashes</code>，<code>Sorted Sets</code>，这五种数据结构先简单介绍下，<code>Strings</code>类型的其实就是我们最常用的 key-value，实际开发中也会用的最多；<code>Lists</code>是列表，这个有些会用来做队列，因为 redis 目前常用的版本支持丰富的列表操作；还有是<code>Sets</code>集合，这个主要的特点就是集合中元素不重复，可以用在有这类需求的场景里；<code>Hashes</code>是叫散列，类似于 Python 中的字典结构；还有就是<code>Sorted Sets</code>这个是个有序集合；一眼看这些其实没啥特别的，除了最后这个有序集合，不过去了解背后的实现方式还是比较有意思的。</p>
|
|
<h2 id="SDS-简单动态字符串"><a href="#SDS-简单动态字符串" class="headerlink" title="SDS 简单动态字符串"></a>SDS 简单动态字符串</h2><p>先从<code>Strings</code>开始说，了解过 C 语言的应该知道，C 语言中的字符串其实是个 <code>char[]</code> 字符数组，redis 也不例外，只是最开始的版本就对这个做了一丢丢的优化，而正是这一丢丢的优化，让这个 redis 的使用效率提升了数倍</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">sdshdr</span> {</span></span><br><span class="line"> <span class="comment">// 字符串长度</span></span><br><span class="line"> <span class="type">int</span> len;</span><br><span class="line"> <span class="comment">// 字符串空余字符数</span></span><br><span class="line"> <span class="type">int</span> <span class="built_in">free</span>;</span><br><span class="line"> <span class="comment">// 字符串内容</span></span><br><span class="line"> <span class="type">char</span> buf[];</span><br><span class="line">};</span><br></pre></td></tr></table></figure>
|
|
<p>这里引用了 redis 在 github 上最早的 2.2 版本的代码，代码路径是<code>https://github.com/antirez/redis/blob/2.2/src/sds.h</code>,可以看到这个结构体里只有仨元素，两个 int 型和一个 char 型数组，两个 int 型其实就是我说的优化，因为 C 语言本身的字符串数组，有两个问题，一个是要知道它实际已被占用的长度，需要去遍历这个数组，第二个就是比较容易踩坑的是遍历的时候要注意它有个以<code>\0</code>作为结尾的特点；通过上面的两个 int 型参数，一个是知道字符串目前的长度，一个是知道字符串还剩余多少位空间，这样子坐着两个操作从 <code>O(N)</code>简化到了<code>O(1)</code>了，还有第二个 free 还有个比较重要的作用就是能防止 C 字符串的溢出问题，在存储之前可以先判断 free 长度，如果长度不够就先扩容了，先介绍到这，这个系列可以写蛮多的，慢慢介绍吧</p>
|
|
<h2 id="链表"><a href="#链表" class="headerlink" title="链表"></a>链表</h2><p>链表是比较常见的数据结构了，但是因为 redis 是用 C 写的，所以在不依赖第三方库的情况下只能自己写一个了，redis 的链表是个有头的链表，而且是无环的，具体的结构我也找了 github 上最早版本的代码</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">listNode</span> {</span></span><br><span class="line"> <span class="comment">// 前置节点</span></span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">listNode</span> *<span class="title">prev</span>;</span></span><br><span class="line"> <span class="comment">// 后置节点</span></span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">listNode</span> *<span class="title">next</span>;</span></span><br><span class="line"> <span class="comment">// 值</span></span><br><span class="line"> <span class="type">void</span> *value;</span><br><span class="line">} listNode;</span><br><span class="line"></span><br><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">list</span> {</span></span><br><span class="line"> <span class="comment">// 链表表头</span></span><br><span class="line"> listNode *head;</span><br><span class="line"> <span class="comment">// 当前节点，也可以说是最后节点</span></span><br><span class="line"> listNode *tail;</span><br><span class="line"> <span class="comment">// 节点复制函数</span></span><br><span class="line"> <span class="type">void</span> *(*dup)(<span class="type">void</span> *ptr);</span><br><span class="line"> <span class="comment">// 节点值释放函数</span></span><br><span class="line"> <span class="type">void</span> (*<span class="built_in">free</span>)(<span class="type">void</span> *ptr);</span><br><span class="line"> <span class="comment">// 节点值比较函数</span></span><br><span class="line"> <span class="type">int</span> (*match)(<span class="type">void</span> *ptr, <span class="type">void</span> *key);</span><br><span class="line"> <span class="comment">// 链表包含的节点数量</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">int</span> len;</span><br><span class="line">} <span class="built_in">list</span>;</span><br></pre></td></tr></table></figure>
|
|
<p>代码地址是这个<code>https://github.com/antirez/redis/blob/2.2/src/adlist.h</code><br>可以看下节点是由listNode承载的，包括值和一个指向前节点跟一个指向后一节点的两个指针，然后值是 void 指针类型，所以可以承载不同类型的值<br>然后是 list结构用来承载一个链表，包含了表头，和表尾，复制函数，释放函数和比较函数，还有链表长度，因为包含了前两个节点，找到表尾节点跟表头都是 <code>O(1)</code>的时间复杂度，还有节点数量，其实这个跟 SDS 是同一个做法，就是空间换时间，这也是写代码里比较常见的做法，以此让一些高频的操作提速。</p>
|
|
<h2 id="字典"><a href="#字典" class="headerlink" title="字典"></a>字典</h2><p>字典也是个常用的数据结构，其实只是叫法不同，数据结构中叫 hash 散列，Java 中叫 Map，PHP 中是数组 array，Python 中也叫字典 dict，因为纯 C 语言本身不带这些数据结构，所以这也是个痛并快乐着的过程，享受 C 语言的高性能的同时也要接受它只提供了语言的基本功能的现实，各种轮子都需要自己造，redis 同样实现了自己的字典<br>下面来看看代码</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">dictEntry</span> {</span></span><br><span class="line"> <span class="type">void</span> *key;</span><br><span class="line"> <span class="type">void</span> *val;</span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">dictEntry</span> *<span class="title">next</span>;</span></span><br><span class="line">} dictEntry;</span><br><span class="line"></span><br><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">dictType</span> {</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="title function_">int</span> <span class="params">(*hashFunction)</span><span class="params">(<span class="type">const</span> <span class="type">void</span> *key)</span>;</span><br><span class="line"> <span class="type">void</span> *(*keyDup)(<span class="type">void</span> *privdata, <span class="type">const</span> <span class="type">void</span> *key);</span><br><span class="line"> <span class="type">void</span> *(*valDup)(<span class="type">void</span> *privdata, <span class="type">const</span> <span class="type">void</span> *obj);</span><br><span class="line"> <span class="type">int</span> (*keyCompare)(<span class="type">void</span> *privdata, <span class="type">const</span> <span class="type">void</span> *key1, <span class="type">const</span> <span class="type">void</span> *key2);</span><br><span class="line"> <span class="type">void</span> (*keyDestructor)(<span class="type">void</span> *privdata, <span class="type">void</span> *key);</span><br><span class="line"> <span class="type">void</span> (*valDestructor)(<span class="type">void</span> *privdata, <span class="type">void</span> *obj);</span><br><span class="line">} dictType;</span><br><span class="line"></span><br><span class="line"><span class="comment">/* This is our hash table structure. Every dictionary has two of this as we</span></span><br><span class="line"><span class="comment"> * implement incremental rehashing, for the old to the new table. */</span></span><br><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">dictht</span> {</span></span><br><span class="line"> dictEntry **table;</span><br><span class="line"> <span class="type">unsigned</span> <span class="type">long</span> size;</span><br><span class="line"> <span class="type">unsigned</span> <span class="type">long</span> sizemask;</span><br><span class="line"> <span class="type">unsigned</span> <span class="type">long</span> used;</span><br><span class="line">} dictht;</span><br><span class="line"></span><br><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">dict</span> {</span></span><br><span class="line"> dictType *type;</span><br><span class="line"> <span class="type">void</span> *privdata;</span><br><span class="line"> dictht ht[<span class="number">2</span>];</span><br><span class="line"> <span class="type">int</span> rehashidx; <span class="comment">/* rehashing not in progress if rehashidx == -1 */</span></span><br><span class="line"> <span class="type">int</span> iterators; <span class="comment">/* number of iterators currently running */</span></span><br><span class="line">} dict;</span><br></pre></td></tr></table></figure>
|
|
<p>看了下这个 2.2 版本的代码跟最新版的其实也差的不是很多，所以还是照旧用老代码，可以看到上面四个结构体中，其实只有三个是存储数据用的，dictType 是用来放操作函数的，那么三个存放数据的结构体分别是干嘛的，这时候感觉需要一个图来说明比较好，稍等，我去画个图~<br><img data-src="https://i.loli.net/2019/12/29/UL4AR1HSEKOh9Qm.png"><br>这个图看着应该比较清楚这些都是用来干嘛的了，dict 是我们的主体结构，它有一个指向 dictType 的指针，这里面包含了字典的操作函数，然后是一个私有数据指针，接下来是一个 dictht 的数组，包含两个dictht，这个就是用来存数据的了，然后是 rehashidx 表示重哈希的状态，当是-1 的时候表示当前没有重哈希，iterators 表示正在遍历的迭代器的数量。<br>首先说说为啥需要有两个 dictht，这是因为字典 dict 这个数据结构随着数据量的增减，会需要在中途做扩容或者缩容操作，如果只有一个的话，对它进行扩容缩容时会影响正常的访问和修改操作，或者说保证正常查询，修改的正确性会比较复杂，并且因为需要高效利用空间，不能一下子申请一个非常大的空间来存很少的数据。当 dict 中 dictht 中的数据量超过 size 的时候负载就超过了 1，就需要进行扩容，这里的其实跟 Java 中的 HashMap 比较类似，超过一定的负载之后进行扩容。这里为啥 size 会超过 1 呢，可能有部分不了解这类结构的同学会比较奇怪，其实就是上图中画的，在数据结构中对于散列的冲突有几类解决方法，比如转换成链表，二次散列，找下个空槽等，这里就使用了链表法，或者说拉链法。当一个新元素通过 hashFunction 得出的 key 跟 sizemask 取模之后的值相同了，那就将其放在原来的节点之前，变成链表挂在数组 dictht.table下面，放在原有节点前是考虑到可能会优先访问。<br>忘了说明下 dictht 跟 dictEntry 的关系了，dictht 就是个哈希表，它里面是个dictEntry 的二维数组，而 dictEntry 是个包含了 key-value 结构之外还有一个 next 指针，因此可以将哈希冲突的以链表的形式保存下来。<br>在重点说下重哈希，可能同样写 Java 的同学对这个比较有感觉，跟 HashMap 一样，会以 2 的 N 次方进行扩容，那么扩容的方法就会比较简单，每个键重哈希要不就在原来这个槽，要不就在原来的槽加原 dictht.size 的位置；然后是重头戏，具体是怎么做扩容呢，其实这里就把第二个 ht 用上了，其实这两个hashtable 的具体作用有点类似于 jvm 中的两个 survival 区，但是又不全一样，因为 redis 在扩容的时候是采用的渐进式地重哈希，什么叫渐进式的呢，就是它不是像 jvm 那种标记复制的模式直接将一个 eden 区和原来的 survival 区存活的对象复制到另一个 survival 区，而是在每一次添加，删除，查找或者更新操作时，都会额外的帮忙搬运一部分的原 dictht 中的数据，这里会根据 rehashidx 的值来判断，如果是-1 表示并没有在重哈希中，如果是 0 表示开始重哈希了，然后rehashidx 还会随着每次的帮忙搬运往上加，但全部被搬运完成后 rehashidx 又变回了-1，又可以扯到Java 中的 Concurrent HashMap, 他在扩容的时候也使用了类似的操作。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Redis</category>
|
|
<category>数据结构</category>
|
|
<category>C</category>
|
|
<category>源码</category>
|
|
<category>Redis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>redis</tag>
|
|
<tag>数据结构</tag>
|
|
<tag>源码</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>redis数据结构介绍三-第三部分 整数集合</title>
|
|
<url>/2020/01/10/redis%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84%E4%BB%8B%E7%BB%8D%E4%B8%89/</url>
|
|
<content><![CDATA[<p>redis中对于 set 其实有两种处理，对于元素均为整型，并且元素数目较少时，使用 intset 作为底层数据结构，否则使用 dict 作为底层数据结构，先看一下代码先</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">intset</span> {</span></span><br><span class="line"> <span class="comment">// 编码方式</span></span><br><span class="line"> <span class="type">uint32_t</span> encoding;</span><br><span class="line"> <span class="comment">// 集合包含的元素数量</span></span><br><span class="line"> <span class="type">uint32_t</span> length;</span><br><span class="line"> <span class="comment">// 保存元素的数组</span></span><br><span class="line"> <span class="type">int8_t</span> contents[];</span><br><span class="line">} intset;</span><br><span class="line"></span><br><span class="line"><span class="comment">/* Note that these encodings are ordered, so:</span></span><br><span class="line"><span class="comment"> * INTSET_ENC_INT16 < INTSET_ENC_INT32 < INTSET_ENC_INT64. */</span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> INTSET_ENC_INT16 (sizeof(int16_t))</span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> INTSET_ENC_INT32 (sizeof(int32_t))</span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> INTSET_ENC_INT64 (sizeof(int64_t))</span></span><br></pre></td></tr></table></figure>
|
|
<p>一眼看，为啥整型还需要编码，然后 int8_t 怎么能存下大整形呢，带着这些疑问，我们一步步分析下去，这里的编码其实指的是这个整型集合里存的究竟是多大的整型，16 位，还是 32 位，还是 64 位，结构体下面的宏定义就是表示了 encoding 的可能取值，INTSET_ENC_INT16 表示每个元素用2个字节存储，INTSET_ENC_INT32 表示每个元素用4个字节存储，INTSET_ENC_INT64 表示每个元素用8个字节存储。因此，intset中存储的整数最多只能占用64bit。length 就是正常的表示集合中元素的数量。最奇怪的应该就是这个 contents 了，是个 int8_t 的数组，那放毛线数据啊，最小的都有 16 位，这里我在看代码和《redis 设计与实现》的时候也有点懵逼，后来查了下发现这是个比较取巧的用法，这里我用自己的理解表述一下，先看看 8，16，32，64 的关系，一眼看就知道都是 2 的 N 次，并且呈两倍关系，而且 8 位刚好一个字节，所以呢其实这里的contents 不是个常规意义上的 int8_t 类型的数组，而是个柔性数组。看下 wiki 的定义</p>
|
|
<blockquote>
|
|
<p>Flexible array members<a href="https://en.wikipedia.org/wiki/Flexible_array_member#cite_note-1">1</a> were introduced in the <a href="https://en.wikipedia.org/wiki/C99">C99</a> standard of the <a href="https://en.wikipedia.org/wiki/C_(programming_language)">C programming language</a> (in particular, in section §6.7.2.1, item 16, page 103).<a href="https://en.wikipedia.org/wiki/Flexible_array_member#cite_note-2">2</a> It is a member of a struct, which is an array without a given dimension. It must be the last member of such a struct and it must be accompanied by at least one other member, as in the following example:</p>
|
|
</blockquote>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">vectord</span> {</span></span><br><span class="line"> <span class="type">size_t</span> len;</span><br><span class="line"> <span class="type">double</span> arr[]; <span class="comment">// the flexible array member must be last</span></span><br><span class="line">};</span><br></pre></td></tr></table></figure>
|
|
<p>在初始化这个 intset 的时候，这个contents数组是不占用空间的，后面的反正用到了申请，那么这里就有一个问题，给出了三种可能的 encoding 值，他们能随便换吗，显然不行，首先在 intset 中数据的存放是有序的，这个有部分原因是方便二分查找，然后存放数据其实随着数据的大小不同会有一个升级的过程，看下图<br><img data-src="https://i.loli.net/2020/01/10/qIc6HgP7wfCLipN.png"><br>新创建的intset只有一个header，总共8个字节。其中encoding = 2, length = 0, 类型都是uint32_t，各占 4 字节，添加15, 5两个元素之后，因为它们是比较小的整数，都能使用2个字节表示，所以encoding不变，值还是2，也就是默认的 <code>INTSET_ENC_INT16</code>，当添加32768的时候，它不再能用2个字节来表示了（2个字节能表达的数据范围是-215~215-1，而32768等于215，超出范围了），因此encoding必须升级到INTSET_ENC_INT32（值为4），即用4个字节表示一个元素。在添加每个元素的过程中，intset始终保持从小到大有序。与ziplist类似，intset也是按小端（little endian）模式存储的（参见维基百科词条<a href="https://en.wikipedia.org/wiki/Endianness">Endianness</a>）。比如，在上图中intset添加完所有数据之后，表示encoding字段的4个字节应该解释成0x00000004，而第4个数据应该解释成0x00008000 = 32768</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Redis</category>
|
|
<category>数据结构</category>
|
|
<category>C</category>
|
|
<category>源码</category>
|
|
<category>Redis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>redis</tag>
|
|
<tag>数据结构</tag>
|
|
<tag>源码</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>redis数据结构介绍二-第二部分 跳表</title>
|
|
<url>/2020/01/04/redis%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84%E4%BB%8B%E7%BB%8D%E4%BA%8C/</url>
|
|
<content><![CDATA[<h2 id="跳表-skiplist"><a href="#跳表-skiplist" class="headerlink" title="跳表 skiplist"></a>跳表 skiplist</h2><p>跳表是个在我们日常的代码中不太常用到的数据结构，相对来讲就没有像数组，链表，字典，散列，树等结构那么熟悉，所以就从头开始分析下，首先是链表，跳表跟链表都有个表字（太硬扯了我🤦♀️），注意这是个有序链表<br><img data-src="https://i.loli.net/2020/01/03/Og9i3pCIfxrMhja.png"><br>如上图，在这个链表里如果我要找到 23，是不是我需要从3，5，9开始一直往后找直到找到 23，也就是说时间复杂度是 O(N),N 的一次幂复杂度，那么我们来看看第二个<br><img data-src="https://i.loli.net/2020/01/03/81P2baupiedOmNf.png"><br>这个结构跟原先有点不一样，它给链表中偶数位的节点又加了一个指针把它们链接起来，这样子当我们要寻找 23 的时候就可以从原来的一个个往下找变成跳着找，先找到 5，然后是 10，接着是 19，然后是 28，这时候发现 28 比 23 大了，那我在退回到 19，然后从下一层原来的链表往前找，<br><img data-src="https://i.loli.net/2020/01/03/NBguAphilKjs2MO.png"><br>这里毛估估是不是前面的节点我就少找了一半,有那么点二分法的意思。<br>前面的其实是跳表的引子，真正的跳表其实不是这样，因为上面的其实有个比较大的问题，就是插入一个元素后需要调整每个元素的指针，在 redis 中的跳表其实是做了个随机层数的优化，因为沿着前面的例子，其实当数据量很大的时候，是不是层数越多，其查询效率越高，但是随着层数变多，要保持这种严格的层数规则其实也会增大处理复杂度，所以 redis 插入每个元素的时候都是使用随机的方式，看一眼代码</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="comment">/* ZSETs use a specialized version of Skiplists */</span></span><br><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">zskiplistNode</span> {</span></span><br><span class="line"> sds ele;</span><br><span class="line"> <span class="type">double</span> score;</span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">zskiplistNode</span> *<span class="title">backward</span>;</span></span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">zskiplistLevel</span> {</span></span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">zskiplistNode</span> *<span class="title">forward</span>;</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">long</span> span;</span><br><span class="line"> } level[];</span><br><span class="line">} zskiplistNode;</span><br><span class="line"></span><br><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">zskiplist</span> {</span></span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">zskiplistNode</span> *<span class="title">header</span>, *<span class="title">tail</span>;</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">long</span> length;</span><br><span class="line"> <span class="type">int</span> level;</span><br><span class="line">} zskiplist;</span><br><span class="line"></span><br><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">zset</span> {</span></span><br><span class="line"> dict *dict;</span><br><span class="line"> zskiplist *zsl;</span><br><span class="line">} zset;</span><br></pre></td></tr></table></figure>
|
|
<p>忘了说了，redis 是把 skiplist 跳表用在 zset 里，zset 是个有序的集合，可以看到 zskiplist 就是个跳表的结构，里面用 header 保存跳表的表头，tail 保存表尾，还有长度和最大层级，具体的跳表节点元素使用 zskiplistNode 表示，里面包含了 sds 类型的元素值，double 类型的分值，用来排序，一个 backward 后向指针和一个 zskiplistLevel 数组，每个 level 包含了一个前向指针，和一个 span，span 表示的是跳表前向指针的跨度，这里再补充一点，前面说了为了灵活这个跳表的新增修改，redis 使用了随机层高的方式插入新节点，但是如果所有节点都随机到很高的层级或者所有都很低的话，跳表的效率优势就会减小，所以 redis 使用了个小技巧，贴下代码</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="meta">#<span class="keyword">define</span> ZSKIPLIST_P 0.25 <span class="comment">/* Skiplist P = 1/4 */</span></span></span><br><span class="line"><span class="type">int</span> <span class="title function_">zslRandomLevel</span><span class="params">(<span class="type">void</span>)</span> {</span><br><span class="line"> <span class="type">int</span> level = <span class="number">1</span>;</span><br><span class="line"> <span class="keyword">while</span> ((random()&<span class="number">0xFFFF</span>) < (ZSKIPLIST_P * <span class="number">0xFFFF</span>))</span><br><span class="line"> level += <span class="number">1</span>;</span><br><span class="line"> <span class="keyword">return</span> (level<ZSKIPLIST_MAXLEVEL) ? level : ZSKIPLIST_MAXLEVEL;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>当随机值跟0xFFFF进行与操作小于ZSKIPLIST_P * 0xFFFF时才会增大 level 的值，因此保持了一个相对递减的概率<br>可以简单分析下，当 random() 的值小于 0xFFFF 的 1/4,才会 level + 1，就意味着当有 1 - 1/4也就是3/4的概率是直接跳出，所以一层的概率是3/4,也就是 1-P，二层的概率是 P*(1-P),三层的概率是 P² * (1-P) 依次递推。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Redis</category>
|
|
<category>数据结构</category>
|
|
<category>C</category>
|
|
<category>源码</category>
|
|
<category>Redis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>redis</tag>
|
|
<tag>数据结构</tag>
|
|
<tag>源码</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>redis数据结构介绍五-第五部分 对象</title>
|
|
<url>/2020/01/20/redis%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84%E4%BB%8B%E7%BB%8D%E4%BA%94/</url>
|
|
<content><![CDATA[<p>前面说了这么些数据结构，其实大家对于 redis 最初的印象应该就是个 key-value 的缓存，类似于 memcache，redis 其实也是个 key-value，key 还是一样的字符串，或者说就是用 redis 自己的动态字符串实现，但是 value 其实就是前面说的那些数据结构，差不多快说完了，还有个 quicklist 后面还有一篇，这里先介绍下 redis 对于这些不同类型的 value 是怎么实现的，首先看下 redisObject 的源码头文件</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="comment">/* The actual Redis Object */</span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> OBJ_STRING 0 <span class="comment">/* String object. */</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> OBJ_LIST 1 <span class="comment">/* List object. */</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> OBJ_SET 2 <span class="comment">/* Set object. */</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> OBJ_ZSET 3 <span class="comment">/* Sorted set object. */</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> OBJ_HASH 4 <span class="comment">/* Hash object. */</span></span></span><br><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment"> * Objects encoding. Some kind of objects like Strings and Hashes can be</span></span><br><span class="line"><span class="comment"> * internally represented in multiple ways. The 'encoding' field of the object</span></span><br><span class="line"><span class="comment"> * is set to one of this fields for this object. */</span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> OBJ_ENCODING_RAW 0 <span class="comment">/* Raw representation */</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> OBJ_ENCODING_INT 1 <span class="comment">/* Encoded as integer */</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> OBJ_ENCODING_HT 2 <span class="comment">/* Encoded as hash table */</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> OBJ_ENCODING_ZIPMAP 3 <span class="comment">/* Encoded as zipmap */</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> OBJ_ENCODING_LINKEDLIST 4 <span class="comment">/* No longer used: old list encoding. */</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> OBJ_ENCODING_ZIPLIST 5 <span class="comment">/* Encoded as ziplist */</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> OBJ_ENCODING_INTSET 6 <span class="comment">/* Encoded as intset */</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> OBJ_ENCODING_SKIPLIST 7 <span class="comment">/* Encoded as skiplist */</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> OBJ_ENCODING_EMBSTR 8 <span class="comment">/* Embedded sds string encoding */</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> OBJ_ENCODING_QUICKLIST 9 <span class="comment">/* Encoded as linked list of ziplists */</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> OBJ_ENCODING_STREAM 10 <span class="comment">/* Encoded as a radix tree of listpacks */</span></span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> LRU_BITS 24</span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> LRU_CLOCK_MAX ((1<span class="string"><<LRU_BITS)-1) /* Max value of obj-></span>lru */</span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> LRU_CLOCK_RESOLUTION 1000 <span class="comment">/* LRU clock resolution in ms */</span></span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> OBJ_SHARED_REFCOUNT INT_MAX</span></span><br><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">redisObject</span> {</span></span><br><span class="line"> <span class="type">unsigned</span> type:<span class="number">4</span>;</span><br><span class="line"> <span class="type">unsigned</span> encoding:<span class="number">4</span>;</span><br><span class="line"> <span class="type">unsigned</span> lru:LRU_BITS; <span class="comment">/* LRU time (relative to global lru_clock) or</span></span><br><span class="line"><span class="comment"> * LFU data (least significant 8 bits frequency</span></span><br><span class="line"><span class="comment"> * and most significant 16 bits access time). */</span></span><br><span class="line"> <span class="type">int</span> refcount;</span><br><span class="line"> <span class="type">void</span> *ptr;</span><br><span class="line">} robj;</span><br></pre></td></tr></table></figure>
|
|
<p>主体结构就是这个 redisObject，</p>
|
|
<ul>
|
|
<li>type： 字段表示对象的类型，它对应的就是 redis 的对外暴露的，或者说用户可以使用的五种类型，OBJ_STRING, OBJ_LIST, OBJ_SET, OBJ_ZSET, OBJ_HASH</li>
|
|
<li>encoding： 字段表示这个对象在 redis 内部的编码方式，由OBJ_ENCODING_开头的 11 种</li>
|
|
<li>lru: 做LRU替换算法用，占24个bit</li>
|
|
<li>refcount: 引用计数。它允许robj对象在某些情况下被共享。</li>
|
|
<li>ptr: 指向底层实现数据结构的指针<br>当 type 是 OBJ_STRING 时，表示类型是个 string，它的编码方式 encoding 可能有 OBJ_ENCODING_RAW，OBJ_ENCODING_INT，OBJ_ENCODING_EMBSTR 三种<br>当 type 是 OBJ_LIST 时，表示类型是 list，它的编码方式 encoding 是 OBJ_ENCODING_QUICKLIST，对于早一些的版本，2.2这种可能还会使用 OBJ_ENCODING_ZIPLIST，OBJ_ENCODING_LINKEDLIST<br>当 type 是 OBJ_SET 时，是个集合，但是得看具体元素的类型，有可能使用整数集合，OBJ_ENCODING_INTSET, 如果元素不全是整型或者数量超过一定限制，那么编码就是 OBJ_ENCODING_HT hash table 了<br>当 type 是 OBJ_ZSET 时，是个有序集合，它底层有可能使用的是 OBJ_ENCODING_ZIPLIST 或者 OBJ_ENCODING_SKIPLIST<br>当 type 是 OBJ_HASH 时，一开始也是 OBJ_ENCODING_ZIPLIST，然后当数据量大于 hash_max_ziplist_entries 时会转成 OBJ_ENCODING_HT</li>
|
|
</ul>
|
|
]]></content>
|
|
<categories>
|
|
<category>Redis</category>
|
|
<category>数据结构</category>
|
|
<category>C</category>
|
|
<category>源码</category>
|
|
<category>Redis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>redis</tag>
|
|
<tag>数据结构</tag>
|
|
<tag>源码</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>redis数据结构介绍六 快表</title>
|
|
<url>/2020/01/22/redis%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84%E4%BB%8B%E7%BB%8D%E5%85%AD/</url>
|
|
<content><![CDATA[<p>这应该是 redis 系列的最后一篇了，讲下快表，其实最前面讲的链表在早先的 redis 版本中也作为 list 的数据结构使用过，但是单纯的链表的缺陷之前也说了，插入便利，但是空间利用率低，并且不能进行二分查找等，检索效率低，ziplist 压缩表的产生也是同理，希望获得更好的性能，包括存储空间和访问性能等，原来我也不懂这个快表要怎么快，然后明白了一个道理，其实并没有什么银弹，只是大牛们会在适合的时候使用最适合的数据结构来实现性能的最大化，这里面有一招就是不同数据结构的组合调整，比如 Java 中的 HashMap，在链表节点数大于 8 时会转变成红黑树，以此提高访问效率，不费话了，回到快表，quicklist，这个数据结构主要使用在 list 类型中，如果我说其实这个 quicklist 就是个链表，可能大家不太会相信，但是事实上的确可以认为 quicklist 是个双向链表，看下代码</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="comment">/* quicklistNode is a 32 byte struct describing a ziplist for a quicklist.</span></span><br><span class="line"><span class="comment"> * We use bit fields keep the quicklistNode at 32 bytes.</span></span><br><span class="line"><span class="comment"> * count: 16 bits, max 65536 (max zl bytes is 65k, so max count actually < 32k).</span></span><br><span class="line"><span class="comment"> * encoding: 2 bits, RAW=1, LZF=2.</span></span><br><span class="line"><span class="comment"> * container: 2 bits, NONE=1, ZIPLIST=2.</span></span><br><span class="line"><span class="comment"> * recompress: 1 bit, bool, true if node is temporarry decompressed for usage.</span></span><br><span class="line"><span class="comment"> * attempted_compress: 1 bit, boolean, used for verifying during testing.</span></span><br><span class="line"><span class="comment"> * extra: 10 bits, free for future use; pads out the remainder of 32 bits */</span></span><br><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">quicklistNode</span> {</span></span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">quicklistNode</span> *<span class="title">prev</span>;</span></span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">quicklistNode</span> *<span class="title">next</span>;</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">char</span> *zl;</span><br><span class="line"> <span class="type">unsigned</span> <span class="type">int</span> sz; <span class="comment">/* ziplist size in bytes */</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">int</span> count : <span class="number">16</span>; <span class="comment">/* count of items in ziplist */</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">int</span> encoding : <span class="number">2</span>; <span class="comment">/* RAW==1 or LZF==2 */</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">int</span> container : <span class="number">2</span>; <span class="comment">/* NONE==1 or ZIPLIST==2 */</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">int</span> recompress : <span class="number">1</span>; <span class="comment">/* was this node previous compressed? */</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">int</span> attempted_compress : <span class="number">1</span>; <span class="comment">/* node can't compress; too small */</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">int</span> extra : <span class="number">10</span>; <span class="comment">/* more bits to steal for future usage */</span></span><br><span class="line">} quicklistNode;</span><br><span class="line"></span><br><span class="line"><span class="comment">/* quicklistLZF is a 4+N byte struct holding 'sz' followed by 'compressed'.</span></span><br><span class="line"><span class="comment"> * 'sz' is byte length of 'compressed' field.</span></span><br><span class="line"><span class="comment"> * 'compressed' is LZF data with total (compressed) length 'sz'</span></span><br><span class="line"><span class="comment"> * <span class="doctag">NOTE:</span> uncompressed length is stored in quicklistNode->sz.</span></span><br><span class="line"><span class="comment"> * When quicklistNode->zl is compressed, node->zl points to a quicklistLZF */</span></span><br><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">quicklistLZF</span> {</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">int</span> sz; <span class="comment">/* LZF size in bytes*/</span></span><br><span class="line"> <span class="type">char</span> compressed[];</span><br><span class="line">} quicklistLZF;</span><br><span class="line"></span><br><span class="line"><span class="comment">/* quicklist is a 40 byte struct (on 64-bit systems) describing a quicklist.</span></span><br><span class="line"><span class="comment"> * 'count' is the number of total entries.</span></span><br><span class="line"><span class="comment"> * 'len' is the number of quicklist nodes.</span></span><br><span class="line"><span class="comment"> * 'compress' is: -1 if compression disabled, otherwise it's the number</span></span><br><span class="line"><span class="comment"> * of quicklistNodes to leave uncompressed at ends of quicklist.</span></span><br><span class="line"><span class="comment"> * 'fill' is the user-requested (or default) fill factor. */</span></span><br><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">quicklist</span> {</span></span><br><span class="line"> quicklistNode *head;</span><br><span class="line"> quicklistNode *tail;</span><br><span class="line"> <span class="type">unsigned</span> <span class="type">long</span> count; <span class="comment">/* total count of all entries in all ziplists */</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">long</span> len; <span class="comment">/* number of quicklistNodes */</span></span><br><span class="line"> <span class="type">int</span> fill : <span class="number">16</span>; <span class="comment">/* fill factor for individual nodes */</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">int</span> compress : <span class="number">16</span>; <span class="comment">/* depth of end nodes not to compress;0=off */</span></span><br><span class="line">} quicklist;</span><br></pre></td></tr></table></figure>
|
|
<p>粗略看下，quicklist 里有 head，tail, quicklistNode里有 prev，next 指针，是不是有链表的基本轮廓了，那么为啥这玩意要称为快表呢，快在哪，关键就在这个<code>unsigned char *zl;</code>zl 是不是前面又看到过，就是 ziplist ，这是什么鬼，链表里用压缩表，这不套娃么，先别急，回顾下前面说的 ziplist，ziplist 有哪些特点，内存利用率高，可以从表头快速定位到尾节点，节点可以从后往前找，但是有个缺点，就是从中间插入的效率比较低，需要整体往后移，这个其实是普通数组的优化版，但还是有数组的一些劣势，所以要真的快，是不是可以将链表跟数组真的结合起来。</p>
|
|
<h2 id="ziplist"><a href="#ziplist" class="headerlink" title="ziplist"></a>ziplist</h2><p>这里有两个 redis 的配置参数，<code>list-max-ziplist-size</code> 和 <code>list-compress-depth</code>，先来说第一个，既然快表是将链表跟压缩表数组结合起来使用，那么具体怎么用呢，比如我有一个 10 个元素的 list，那具体怎么放，每个 quicklistNode 里放多大的 ziplist，假如每个快表节点的 ziplist 只放一个元素，那么其实这就退化成了一个链表，如果 10 个元素放在一个 quicklistNode 的 ziplist 里，那就退化成了一个 ziplist，所以有了这个 <code>list-max-ziplist-size</code>,而且它还比较牛，能取正负值，当是正值时，对应的就是每个 quicklistNode 的 ziplist 中的元素个数，比如配置了 <code>list-max-ziplist-size = 5</code>，那么我刚才的 10 个元素的 list 就是一个两个 quicklistNode 组成的快表，每个 quicklistNode 中的 ziplist 包含了五个元素，当 <code>list-max-ziplist-size</code>取负值的时候，它限制了 ziplist 的字节数</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">size_t offset = (-fill) - 1;</span><br><span class="line">if (offset < (sizeof(optimization_level) / sizeof(*optimization_level))) {</span><br><span class="line"> if (sz <= optimization_level[offset]) {</span><br><span class="line"> return 1;</span><br><span class="line"> } else {</span><br><span class="line"> return 0;</span><br><span class="line"> }</span><br><span class="line">} else {</span><br><span class="line"> return 0;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">/* Optimization levels for size-based filling */</span><br><span class="line">static const size_t optimization_level[] = {4096, 8192, 16384, 32768, 65536};</span><br><span class="line"></span><br><span class="line">/* Create a new quicklist.</span><br><span class="line"> * Free with quicklistRelease(). */</span><br><span class="line">quicklist *quicklistCreate(void) {</span><br><span class="line"> struct quicklist *quicklist;</span><br><span class="line"></span><br><span class="line"> quicklist = zmalloc(sizeof(*quicklist));</span><br><span class="line"> quicklist->head = quicklist->tail = NULL;</span><br><span class="line"> quicklist->len = 0;</span><br><span class="line"> quicklist->count = 0;</span><br><span class="line"> quicklist->compress = 0;</span><br><span class="line"> quicklist->fill = -2;</span><br><span class="line"> return quicklist;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这个 fill 就是传进来的 <code>list-max-ziplist-size</code>, 具体对应的就是</p>
|
|
<ul>
|
|
<li>-5: 每个quicklist节点上的ziplist大小不能超过64 Kb。（注：1kb => 1024 bytes）</li>
|
|
<li>-4: 每个quicklist节点上的ziplist大小不能超过32 Kb。</li>
|
|
<li>-3: 每个quicklist节点上的ziplist大小不能超过16 Kb。</li>
|
|
<li>-2: 每个quicklist节点上的ziplist大小不能超过8 Kb。（-2是Redis给出的默认值）也就是上面的 <code>quicklist->fill = -2;</code></li>
|
|
<li>-1: 每个quicklist节点上的ziplist大小不能超过4 Kb。</li>
|
|
</ul>
|
|
<h2 id="压缩"><a href="#压缩" class="headerlink" title="压缩"></a>压缩</h2><p><code>list-compress-depth</code>这个参数呢是用来配置压缩的，等等压缩是为啥，不是里面已经是压缩表了么，大牛们就是为了性能殚精竭虑，这里考虑到的是一个场景，一般状况下，list 都是两端的访问频率比较高，那么是不是可以对中间的数据进行压缩，那么这个参数就是用来表示</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">/* depth of end nodes not to compress;0=off */</span><br></pre></td></tr></table></figure>
|
|
<ul>
|
|
<li>0，代表不压缩，默认值</li>
|
|
<li>1，两端各一个节点不压缩</li>
|
|
<li>2，两端各两个节点不压缩</li>
|
|
<li>… 依次类推<br>压缩后的 ziplist 就会变成 quicklistLZF，然后替换 zl 指针，这里使用的是 <a href="http://oldhome.schmorp.de/marc/liblzf.html">LZF</a> 压缩算法，压缩后的 quicklistLZF 中的 compressed 也是个柔性数组，压缩后的 ziplist 整个就放进这个柔性数组</li>
|
|
</ul>
|
|
<h2 id="插入过程"><a href="#插入过程" class="headerlink" title="插入过程"></a>插入过程</h2><p>简单说下插入元素的过程</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">/* Wrapper to allow argument-based switching between HEAD/TAIL pop */</span><br><span class="line">void quicklistPush(quicklist *quicklist, void *value, const size_t sz,</span><br><span class="line"> int where) {</span><br><span class="line"> if (where == QUICKLIST_HEAD) {</span><br><span class="line"> quicklistPushHead(quicklist, value, sz);</span><br><span class="line"> } else if (where == QUICKLIST_TAIL) {</span><br><span class="line"> quicklistPushTail(quicklist, value, sz);</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">/* Add new entry to head node of quicklist.</span><br><span class="line"> *</span><br><span class="line"> * Returns 0 if used existing head.</span><br><span class="line"> * Returns 1 if new head created. */</span><br><span class="line">int quicklistPushHead(quicklist *quicklist, void *value, size_t sz) {</span><br><span class="line"> quicklistNode *orig_head = quicklist->head;</span><br><span class="line"> if (likely(</span><br><span class="line"> _quicklistNodeAllowInsert(quicklist->head, quicklist->fill, sz))) {</span><br><span class="line"> quicklist->head->zl =</span><br><span class="line"> ziplistPush(quicklist->head->zl, value, sz, ZIPLIST_HEAD);</span><br><span class="line"> quicklistNodeUpdateSz(quicklist->head);</span><br><span class="line"> } else {</span><br><span class="line"> quicklistNode *node = quicklistCreateNode();</span><br><span class="line"> node->zl = ziplistPush(ziplistNew(), value, sz, ZIPLIST_HEAD);</span><br><span class="line"></span><br><span class="line"> quicklistNodeUpdateSz(node);</span><br><span class="line"> _quicklistInsertNodeBefore(quicklist, quicklist->head, node);</span><br><span class="line"> }</span><br><span class="line"> quicklist->count++;</span><br><span class="line"> quicklist->head->count++;</span><br><span class="line"> return (orig_head != quicklist->head);</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">/* Add new entry to tail node of quicklist.</span><br><span class="line"> *</span><br><span class="line"> * Returns 0 if used existing tail.</span><br><span class="line"> * Returns 1 if new tail created. */</span><br><span class="line">int quicklistPushTail(quicklist *quicklist, void *value, size_t sz) {</span><br><span class="line"> quicklistNode *orig_tail = quicklist->tail;</span><br><span class="line"> if (likely(</span><br><span class="line"> _quicklistNodeAllowInsert(quicklist->tail, quicklist->fill, sz))) {</span><br><span class="line"> quicklist->tail->zl =</span><br><span class="line"> ziplistPush(quicklist->tail->zl, value, sz, ZIPLIST_TAIL);</span><br><span class="line"> quicklistNodeUpdateSz(quicklist->tail);</span><br><span class="line"> } else {</span><br><span class="line"> quicklistNode *node = quicklistCreateNode();</span><br><span class="line"> node->zl = ziplistPush(ziplistNew(), value, sz, ZIPLIST_TAIL);</span><br><span class="line"></span><br><span class="line"> quicklistNodeUpdateSz(node);</span><br><span class="line"> _quicklistInsertNodeAfter(quicklist, quicklist->tail, node);</span><br><span class="line"> }</span><br><span class="line"> quicklist->count++;</span><br><span class="line"> quicklist->tail->count++;</span><br><span class="line"> return (orig_tail != quicklist->tail);</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">/* Wrappers for node inserting around existing node. */</span><br><span class="line">REDIS_STATIC void _quicklistInsertNodeBefore(quicklist *quicklist,</span><br><span class="line"> quicklistNode *old_node,</span><br><span class="line"> quicklistNode *new_node) {</span><br><span class="line"> __quicklistInsertNode(quicklist, old_node, new_node, 0);</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">REDIS_STATIC void _quicklistInsertNodeAfter(quicklist *quicklist,</span><br><span class="line"> quicklistNode *old_node,</span><br><span class="line"> quicklistNode *new_node) {</span><br><span class="line"> __quicklistInsertNode(quicklist, old_node, new_node, 1);</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">/* Insert 'new_node' after 'old_node' if 'after' is 1.</span><br><span class="line"> * Insert 'new_node' before 'old_node' if 'after' is 0.</span><br><span class="line"> * Note: 'new_node' is *always* uncompressed, so if we assign it to</span><br><span class="line"> * head or tail, we do not need to uncompress it. */</span><br><span class="line">REDIS_STATIC void __quicklistInsertNode(quicklist *quicklist,</span><br><span class="line"> quicklistNode *old_node,</span><br><span class="line"> quicklistNode *new_node, int after) {</span><br><span class="line"> if (after) {</span><br><span class="line"> new_node->prev = old_node;</span><br><span class="line"> if (old_node) {</span><br><span class="line"> new_node->next = old_node->next;</span><br><span class="line"> if (old_node->next)</span><br><span class="line"> old_node->next->prev = new_node;</span><br><span class="line"> old_node->next = new_node;</span><br><span class="line"> }</span><br><span class="line"> if (quicklist->tail == old_node)</span><br><span class="line"> quicklist->tail = new_node;</span><br><span class="line"> } else {</span><br><span class="line"> new_node->next = old_node;</span><br><span class="line"> if (old_node) {</span><br><span class="line"> new_node->prev = old_node->prev;</span><br><span class="line"> if (old_node->prev)</span><br><span class="line"> old_node->prev->next = new_node;</span><br><span class="line"> old_node->prev = new_node;</span><br><span class="line"> }</span><br><span class="line"> if (quicklist->head == old_node)</span><br><span class="line"> quicklist->head = new_node;</span><br><span class="line"> }</span><br><span class="line"> /* If this insert creates the only element so far, initialize head/tail. */</span><br><span class="line"> if (quicklist->len == 0) {</span><br><span class="line"> quicklist->head = quicklist->tail = new_node;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> if (old_node)</span><br><span class="line"> quicklistCompress(quicklist, old_node);</span><br><span class="line"></span><br><span class="line"> quicklist->len++;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>前面第一步先根据插入的是头还是尾选择不同的 push 函数，quicklistPushHead 或者 quicklistPushTail，举例分析下从头插入的 quicklistPushHead，先判断当前的 quicklistNode 节点还能不能允许再往 ziplist 里添加元素，如果可以就添加，如果不允许就新建一个 quicklistNode，然后调用 _quicklistInsertNodeBefore 将节点插进去，具体插入quicklist节点的操作类似链表的插入。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Redis</category>
|
|
<category>数据结构</category>
|
|
<category>C</category>
|
|
<category>源码</category>
|
|
<category>Redis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>redis</tag>
|
|
<tag>数据结构</tag>
|
|
<tag>源码</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>redis数据结构介绍四-第四部分 压缩表</title>
|
|
<url>/2020/01/19/redis%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84%E4%BB%8B%E7%BB%8D%E5%9B%9B/</url>
|
|
<content><![CDATA[<p>在 redis 中还有一类表型数据结构叫压缩表，ziplist，它的目的是替代链表，链表是个很容易理解的数据结构，双向链表有前后指针，有带头结点的有的不带，但是链表有个比较大的问题是相对于普通的数组，它的内存不连续，碎片化的存储，内存利用效率不高，而且指针寻址相对于直接使用偏移量的话，也有一定的效率劣势，当然这不是主要的原因，ziplist 设计的主要目的是让链表的内存使用更高效</p>
|
|
<blockquote>
|
|
<p>The ziplist is a specially encoded dually linked list that is designed to be very memory efficient.<br>这是摘自 redis 源码中ziplist.c 文件的注释，也说明了原因，它的大概结构是这样子</p>
|
|
</blockquote>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line"><zlbytes> <zltail> <zllen> <entry> <entry> ... <entry> <zlend></span><br></pre></td></tr></table></figure>
|
|
<p>其中<br><code><zlbytes></code>表示 ziplist 占用的字节总数，类型是uint32_t，32 位的无符号整型，当然表示的字节数也包含自己本身占用的 4 个<br><code><zltail></code> 类型也是是uint32_t，表示ziplist表中最后一项（entry）在ziplist中的偏移字节数。<code><zltail></code>的存在，使得我们可以很方便地找到最后一项（不用遍历整个ziplist），从而可以在ziplist尾端快速地执行push或pop操作。<br><code><uint16_t zllen></code> 表示ziplist 中的数据项个数，因为是 16 位，所以当数量超过所能表示的最大的数量，它的 16 位全会置为 1，但是真实的数量需要遍历整个 ziplist 才能知道<br><code><entry></code>是具体的数据项，后面解释<br><code><zlend></code> ziplist 的最后一个字节，固定是255。<br>再看一下<code><entry></code>中的具体结构，</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line"><prevlen> <encoding> <entry-data></span><br></pre></td></tr></table></figure>
|
|
<p>首先这个<code><prevlen></code>有两种情况，一种是前面的元素的长度，如果是小于等于 253的时候就用一个uint8_t 来表示前一元素的长度，如果大于的话他将占用五个字节，第一个字节是 254，即表示这个字节已经表示不下了，需要后面的四个字节帮忙表示<br><code><encoding></code>这个就比较复杂，把源码的注释放下面先看下</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">* |00pppppp| - 1 byte</span><br><span class="line">* String value with length less than or equal to 63 bytes (6 bits).</span><br><span class="line">* "pppppp" represents the unsigned 6 bit length.</span><br><span class="line">* |01pppppp|qqqqqqqq| - 2 bytes</span><br><span class="line">* String value with length less than or equal to 16383 bytes (14 bits).</span><br><span class="line">* IMPORTANT: The 14 bit number is stored in big endian.</span><br><span class="line">* |10000000|qqqqqqqq|rrrrrrrr|ssssssss|tttttttt| - 5 bytes</span><br><span class="line">* String value with length greater than or equal to 16384 bytes.</span><br><span class="line">* Only the 4 bytes following the first byte represents the length</span><br><span class="line">* up to 32^2-1. The 6 lower bits of the first byte are not used and</span><br><span class="line">* are set to zero.</span><br><span class="line">* IMPORTANT: The 32 bit number is stored in big endian.</span><br><span class="line">* |11000000| - 3 bytes</span><br><span class="line">* Integer encoded as int16_t (2 bytes).</span><br><span class="line">* |11010000| - 5 bytes</span><br><span class="line">* Integer encoded as int32_t (4 bytes).</span><br><span class="line">* |11100000| - 9 bytes</span><br><span class="line">* Integer encoded as int64_t (8 bytes).</span><br><span class="line">* |11110000| - 4 bytes</span><br><span class="line">* Integer encoded as 24 bit signed (3 bytes).</span><br><span class="line">* |11111110| - 2 bytes</span><br><span class="line">* Integer encoded as 8 bit signed (1 byte).</span><br><span class="line">* |1111xxxx| - (with xxxx between 0000 and 1101) immediate 4 bit integer.</span><br><span class="line">* Unsigned integer from 0 to 12. The encoded value is actually from</span><br><span class="line">* 1 to 13 because 0000 and 1111 can not be used, so 1 should be</span><br><span class="line">* subtracted from the encoded 4 bit value to obtain the right value.</span><br><span class="line">* |11111111| - End of ziplist special entry.</span><br></pre></td></tr></table></figure>
|
|
<p>首先如果 encoding 的前两位是 00 的话代表这个元素是个 6 位的字符串，即直接将数据保存在 encoding 中，不消耗额外的<code><entry-data></code>，如果前两位是 01 的话表示是个 14 位的字符串，如果是 10 的话表示encoding 块之后的四个字节是存放字符串类型的数据，encoding 的剩余 6 位置 0。<br>如果 encoding 的前两位是 11 的话表示这是个整型，具体的如果后两位是00的话，表示后面是个2字节的 int16_t 类型，如果是01的话，后面是个4字节的int32_t,如果是10的话后面是8字节的int64_t,如果是 11 的话后面是 3 字节的有符号整型，这些都要最后 4 位都是 0 的情况噢<br>剩下当是<code>11111110</code>时，则表示是一个1 字节的有符号数，如果是 <code>1111xxxx</code>，其中<code>xxxx</code>在0000 到 1101 表示实际的 1 到 13，为啥呢，因为 0000 前面已经用过了，而 1110 跟 1111 也都有用了。<br>看个具体的例子(上下有点对不齐，将就看)</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">[0f 00 00 00] [0c 00 00 00] [02 00] [00 f3] [02 f6] [ff]</span><br><span class="line">|**zlbytes***| |***zltail***| |*zllen*| |entry1 entry2| |zlend|</span><br></pre></td></tr></table></figure>
|
|
<p>第一部分代表整个 ziplist 有 15 个字节，zlbytes 自己占了 4 个 zltail 表示最后一个元素的偏移量，第 13 个字节起，zllen 表示有 2 个元素，第一个元素是<code>00f3</code>,00表示前一个元素长度是 0，本来前面就没元素(不过不知道这个能不能优化这一字节)，然后是 f3，换成二进制就是11110011,对照上面的注释，是落在|1111xxxx|这个类型里，注意这个其实是用 0001 到 1101 也就是 1到 13 来表示 0到 12，所以 f3 应该就是 2，第一个元素是 2，第二个元素呢，02 代表前一个元素也就是刚才说的这个，占用 2 字节，f6 展开也是刚才的类型，实际是 5，ff 表示 ziplist 的结尾，所以这个 ziplist 里面是两个元素，2 跟 5</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Redis</category>
|
|
<category>数据结构</category>
|
|
<category>C</category>
|
|
<category>源码</category>
|
|
<category>Redis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>redis</tag>
|
|
<tag>数据结构</tag>
|
|
<tag>源码</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>redis淘汰策略复习</title>
|
|
<url>/2021/08/01/redis%E6%B7%98%E6%B1%B0%E7%AD%96%E7%95%A5%E5%A4%8D%E4%B9%A0/</url>
|
|
<content><![CDATA[<p>前面复习了 redis 的过期策略，这里再复习下淘汰策略，淘汰跟过期的区别有时候会被混淆了，过期主要针对那些设置了过期时间的 key，应该说是一种逻辑策略，是主动的还是被动的加定时的，两种有各自的取舍，而淘汰也可以看成是一种保持系统稳定的策略，因为如果内存满了，不采取任何策略处理，那大概率会导致系统故障，之前其实主要从源码角度分析过redis 的 LRU 和 LFU，但这个是偏底层的实现，抠得比较细，那么具体的系统层面的配置是有哪些策略，来看下 redis labs 的介绍</p>
|
|
<table>
|
|
<thead>
|
|
<tr>
|
|
<th align="center"><strong>Policy</strong></th>
|
|
<th align="center"><strong>Description</strong></th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr>
|
|
<td align="center">noeviction 不逐出</td>
|
|
<td align="center">Returns an error if the memory limit has been reached when trying to insert more data，插入更多数据时，如果内存达到上限了，返回错误</td>
|
|
</tr>
|
|
<tr>
|
|
<td align="center">allkeys-lru 所有的 key 使用 lru 逐出</td>
|
|
<td align="center">Evicts the least recently used keys out of all keys 在所有 key 中逐出最近最少使用的</td>
|
|
</tr>
|
|
<tr>
|
|
<td align="center">allkeys-lfu 所有的 key 使用 lfu 逐出</td>
|
|
<td align="center">Evicts the least frequently used keys out of all keys 在所有 key 中逐出最近最不频繁使用的</td>
|
|
</tr>
|
|
<tr>
|
|
<td align="center">allkeys-random 所有的 key 中随机逐出</td>
|
|
<td align="center">Randomly evicts keys out of all keys 在所有 key 中随机逐出</td>
|
|
</tr>
|
|
<tr>
|
|
<td align="center">volatile-lru</td>
|
|
<td align="center">Evicts the least recently used keys out of all keys with an “expire” field set 在设置了过期时间的 key 空间 expire 中使用 lru 策略逐出</td>
|
|
</tr>
|
|
<tr>
|
|
<td align="center">volatile-lfu</td>
|
|
<td align="center">Evicts the least frequently used keys out of all keys with an “expire” field set 在设置了过期时间的 key 空间 expire 中使用 lfu 策略逐出</td>
|
|
</tr>
|
|
<tr>
|
|
<td align="center">volatile-random</td>
|
|
<td align="center">Randomly evicts keys with an “expire” field set 在设置了过期时间的 key 空间 expire 中随机逐出</td>
|
|
</tr>
|
|
<tr>
|
|
<td align="center">volatile-ttl</td>
|
|
<td align="center">Evicts the shortest time-to-live keys out of all keys with an “expire” field set.在设置了过期时间的 key 空间 expire 中逐出更早过期的</td>
|
|
</tr>
|
|
</tbody></table>
|
|
<p>而在这其中默认使用的策略是 volatile-lru，对 lru 跟 lfu 想有更多的了解可以看下我之前的文章<a href="https://nicksxs.me/2020/04/18/redis%E7%B3%BB%E5%88%97%E4%BB%8B%E7%BB%8D%E5%85%AB/">redis系列介绍八-淘汰策略</a></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>redis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>redis</tag>
|
|
<tag>淘汰策略</tag>
|
|
<tag>应用</tag>
|
|
<tag>Evict</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>redis系列介绍七-过期策略</title>
|
|
<url>/2020/04/12/redis%E7%B3%BB%E5%88%97%E4%BB%8B%E7%BB%8D%E4%B8%83/</url>
|
|
<content><![CDATA[<p>这一篇不再是数据结构介绍了，大致的数据结构基本都介绍了，这一篇主要是查漏补缺，或者说讲一些重要且基本的概念，也可能是经常被忽略的，很多讲 redis 的系列文章可能都会忽略，学习 redis 的时候也会，因为觉得源码学习就是讲主要的数据结构和“算法”学习了就好了。<br>redis 的主要应用就是拿来作为高性能的缓存，那么缓存一般有些啥需要注意的，首先是访问速度，如果取得跟数据库一样快，那就没什么存在的意义，第二个是缓存的字面意思，我只是为了让数据读取快一些，通常大部分的场景这个是需要更新过期的，这里就把我要讲的第一点引出来了（真累，</p>
|
|
<h2 id="redis过期策略"><a href="#redis过期策略" class="headerlink" title="redis过期策略"></a>redis过期策略</h2><p>redis 是如何过期缓存的，可以猜测下，最无脑的就是每个设置了过期时间的 key 都设个定时器，过期了就删除，这种显然消耗太大，清理地最及时，还有的就是 redis 正在采用的懒汉清理策略和定期清理<br>懒汉策略就是在使用的时候去检查缓存是否过期，比如 get 操作时，先判断下这个 key 是否已经过期了，如果过期了就删掉，并且返回空，如果没过期则正常返回<br>主要代码是</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="comment">/* This function is called when we are going to perform some operation</span></span><br><span class="line"><span class="comment"> * in a given key, but such key may be already logically expired even if</span></span><br><span class="line"><span class="comment"> * it still exists in the database. The main way this function is called</span></span><br><span class="line"><span class="comment"> * is via lookupKey*() family of functions.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * The behavior of the function depends on the replication role of the</span></span><br><span class="line"><span class="comment"> * instance, because slave instances do not expire keys, they wait</span></span><br><span class="line"><span class="comment"> * for DELs from the master for consistency matters. However even</span></span><br><span class="line"><span class="comment"> * slaves will try to have a coherent return value for the function,</span></span><br><span class="line"><span class="comment"> * so that read commands executed in the slave side will be able to</span></span><br><span class="line"><span class="comment"> * behave like if the key is expired even if still present (because the</span></span><br><span class="line"><span class="comment"> * master has yet to propagate the DEL).</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * In masters as a side effect of finding a key which is expired, such</span></span><br><span class="line"><span class="comment"> * key will be evicted from the database. Also this may trigger the</span></span><br><span class="line"><span class="comment"> * propagation of a DEL/UNLINK command in AOF / replication stream.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * The return value of the function is 0 if the key is still valid,</span></span><br><span class="line"><span class="comment"> * otherwise the function returns 1 if the key is expired. */</span></span><br><span class="line"><span class="type">int</span> <span class="title function_">expireIfNeeded</span><span class="params">(redisDb *db, robj *key)</span> {</span><br><span class="line"> <span class="keyword">if</span> (!keyIsExpired(db,key)) <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* If we are running in the context of a slave, instead of</span></span><br><span class="line"><span class="comment"> * evicting the expired key from the database, we return ASAP:</span></span><br><span class="line"><span class="comment"> * the slave key expiration is controlled by the master that will</span></span><br><span class="line"><span class="comment"> * send us synthesized DEL operations for expired keys.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * Still we try to return the right information to the caller,</span></span><br><span class="line"><span class="comment"> * that is, 0 if we think the key should be still valid, 1 if</span></span><br><span class="line"><span class="comment"> * we think the key is expired at this time. */</span></span><br><span class="line"> <span class="keyword">if</span> (server.masterhost != <span class="literal">NULL</span>) <span class="keyword">return</span> <span class="number">1</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Delete the key */</span></span><br><span class="line"> server.stat_expiredkeys++;</span><br><span class="line"> propagateExpire(db,key,server.lazyfree_lazy_expire);</span><br><span class="line"> notifyKeyspaceEvent(NOTIFY_EXPIRED,</span><br><span class="line"> <span class="string">"expired"</span>,key,db->id);</span><br><span class="line"> <span class="keyword">return</span> server.lazyfree_lazy_expire ? dbAsyncDelete(db,key) :</span><br><span class="line"> dbSyncDelete(db,key);</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">/* Check if the key is expired. */</span></span><br><span class="line"><span class="type">int</span> <span class="title function_">keyIsExpired</span><span class="params">(redisDb *db, robj *key)</span> {</span><br><span class="line"> <span class="type">mstime_t</span> when = getExpire(db,key);</span><br><span class="line"> <span class="type">mstime_t</span> now;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (when < <span class="number">0</span>) <span class="keyword">return</span> <span class="number">0</span>; <span class="comment">/* No expire for this key */</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Don't expire anything while loading. It will be done later. */</span></span><br><span class="line"> <span class="keyword">if</span> (server.loading) <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* If we are in the context of a Lua script, we pretend that time is</span></span><br><span class="line"><span class="comment"> * blocked to when the Lua script started. This way a key can expire</span></span><br><span class="line"><span class="comment"> * only the first time it is accessed and not in the middle of the</span></span><br><span class="line"><span class="comment"> * script execution, making propagation to slaves / AOF consistent.</span></span><br><span class="line"><span class="comment"> * See issue #1525 on Github for more information. */</span></span><br><span class="line"> <span class="keyword">if</span> (server.lua_caller) {</span><br><span class="line"> now = server.lua_time_start;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">/* If we are in the middle of a command execution, we still want to use</span></span><br><span class="line"><span class="comment"> * a reference time that does not change: in that case we just use the</span></span><br><span class="line"><span class="comment"> * cached time, that we update before each call in the call() function.</span></span><br><span class="line"><span class="comment"> * This way we avoid that commands such as RPOPLPUSH or similar, that</span></span><br><span class="line"><span class="comment"> * may re-open the same key multiple times, can invalidate an already</span></span><br><span class="line"><span class="comment"> * open object in a next call, if the next call will see the key expired,</span></span><br><span class="line"><span class="comment"> * while the first did not. */</span></span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span> (server.fixed_time_expire > <span class="number">0</span>) {</span><br><span class="line"> now = server.mstime;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">/* For the other cases, we want to use the most fresh time we have. */</span></span><br><span class="line"> <span class="keyword">else</span> {</span><br><span class="line"> now = mstime();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* The key expired if the current (virtual or real) time is greater</span></span><br><span class="line"><span class="comment"> * than the expire time of the key. */</span></span><br><span class="line"> <span class="keyword">return</span> now > when;</span><br><span class="line">}</span><br><span class="line"><span class="comment">/* Return the expire time of the specified key, or -1 if no expire</span></span><br><span class="line"><span class="comment"> * is associated with this key (i.e. the key is non volatile) */</span></span><br><span class="line"><span class="type">long</span> <span class="type">long</span> <span class="title function_">getExpire</span><span class="params">(redisDb *db, robj *key)</span> {</span><br><span class="line"> dictEntry *de;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* No expire? return ASAP */</span></span><br><span class="line"> <span class="keyword">if</span> (dictSize(db->expires) == <span class="number">0</span> ||</span><br><span class="line"> (de = dictFind(db->expires,key->ptr)) == <span class="literal">NULL</span>) <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* The entry was found in the expire dict, this means it should also</span></span><br><span class="line"><span class="comment"> * be present in the main dict (safety check). */</span></span><br><span class="line"> serverAssertWithInfo(<span class="literal">NULL</span>,key,dictFind(db->dict,key->ptr) != <span class="literal">NULL</span>);</span><br><span class="line"> <span class="keyword">return</span> dictGetSignedIntegerVal(de);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里有几点要注意的，第一是当惰性删除时会根据lazyfree_lazy_expire这个参数去判断是执行同步删除还是异步删除，另外一点是对于 slave，是不需要执行的，因为会在 master 过期时向 slave 发送 del 指令。<br>光采用这个策略会有什么问题呢，假如一些key 一直未被访问，那这些 key 就不会过期了，导致一直被占用着内存，所以 redis 采取了懒汉式过期加定期过期策略，定期策略是怎么执行的呢</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="comment">/* This function handles 'background' operations we are required to do</span></span><br><span class="line"><span class="comment"> * incrementally in Redis databases, such as active key expiring, resizing,</span></span><br><span class="line"><span class="comment"> * rehashing. */</span></span><br><span class="line"><span class="type">void</span> <span class="title function_">databasesCron</span><span class="params">(<span class="type">void</span>)</span> {</span><br><span class="line"> <span class="comment">/* Expire keys by random sampling. Not required for slaves</span></span><br><span class="line"><span class="comment"> * as master will synthesize DELs for us. */</span></span><br><span class="line"> <span class="keyword">if</span> (server.active_expire_enabled) {</span><br><span class="line"> <span class="keyword">if</span> (server.masterhost == <span class="literal">NULL</span>) {</span><br><span class="line"> activeExpireCycle(ACTIVE_EXPIRE_CYCLE_SLOW);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> expireSlaveKeys();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Defrag keys gradually. */</span></span><br><span class="line"> activeDefragCycle();</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Perform hash tables rehashing if needed, but only if there are no</span></span><br><span class="line"><span class="comment"> * other processes saving the DB on disk. Otherwise rehashing is bad</span></span><br><span class="line"><span class="comment"> * as will cause a lot of copy-on-write of memory pages. */</span></span><br><span class="line"> <span class="keyword">if</span> (!hasActiveChildProcess()) {</span><br><span class="line"> <span class="comment">/* We use global counters so if we stop the computation at a given</span></span><br><span class="line"><span class="comment"> * DB we'll be able to start from the successive in the next</span></span><br><span class="line"><span class="comment"> * cron loop iteration. */</span></span><br><span class="line"> <span class="type">static</span> <span class="type">unsigned</span> <span class="type">int</span> resize_db = <span class="number">0</span>;</span><br><span class="line"> <span class="type">static</span> <span class="type">unsigned</span> <span class="type">int</span> rehash_db = <span class="number">0</span>;</span><br><span class="line"> <span class="type">int</span> dbs_per_call = CRON_DBS_PER_CALL;</span><br><span class="line"> <span class="type">int</span> j;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Don't test more DBs than we have. */</span></span><br><span class="line"> <span class="keyword">if</span> (dbs_per_call > server.dbnum) dbs_per_call = server.dbnum;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Resize */</span></span><br><span class="line"> <span class="keyword">for</span> (j = <span class="number">0</span>; j < dbs_per_call; j++) {</span><br><span class="line"> tryResizeHashTables(resize_db % server.dbnum);</span><br><span class="line"> resize_db++;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Rehash */</span></span><br><span class="line"> <span class="keyword">if</span> (server.activerehashing) {</span><br><span class="line"> <span class="keyword">for</span> (j = <span class="number">0</span>; j < dbs_per_call; j++) {</span><br><span class="line"> <span class="type">int</span> work_done = incrementallyRehash(rehash_db);</span><br><span class="line"> <span class="keyword">if</span> (work_done) {</span><br><span class="line"> <span class="comment">/* If the function did some work, stop here, we'll do</span></span><br><span class="line"><span class="comment"> * more at the next cron loop. */</span></span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">/* If this db didn't need rehash, we'll try the next one. */</span></span><br><span class="line"> rehash_db++;</span><br><span class="line"> rehash_db %= server.dbnum;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"><span class="comment">/* Try to expire a few timed out keys. The algorithm used is adaptive and</span></span><br><span class="line"><span class="comment"> * will use few CPU cycles if there are few expiring keys, otherwise</span></span><br><span class="line"><span class="comment"> * it will get more aggressive to avoid that too much memory is used by</span></span><br><span class="line"><span class="comment"> * keys that can be removed from the keyspace.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * Every expire cycle tests multiple databases: the next call will start</span></span><br><span class="line"><span class="comment"> * again from the next db, with the exception of exists for time limit: in that</span></span><br><span class="line"><span class="comment"> * case we restart again from the last database we were processing. Anyway</span></span><br><span class="line"><span class="comment"> * no more than CRON_DBS_PER_CALL databases are tested at every iteration.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * The function can perform more or less work, depending on the "type"</span></span><br><span class="line"><span class="comment"> * argument. It can execute a "fast cycle" or a "slow cycle". The slow</span></span><br><span class="line"><span class="comment"> * cycle is the main way we collect expired cycles: this happens with</span></span><br><span class="line"><span class="comment"> * the "server.hz" frequency (usually 10 hertz).</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * However the slow cycle can exit for timeout, since it used too much time.</span></span><br><span class="line"><span class="comment"> * For this reason the function is also invoked to perform a fast cycle</span></span><br><span class="line"><span class="comment"> * at every event loop cycle, in the beforeSleep() function. The fast cycle</span></span><br><span class="line"><span class="comment"> * will try to perform less work, but will do it much more often.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * The following are the details of the two expire cycles and their stop</span></span><br><span class="line"><span class="comment"> * conditions:</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * If type is ACTIVE_EXPIRE_CYCLE_FAST the function will try to run a</span></span><br><span class="line"><span class="comment"> * "fast" expire cycle that takes no longer than EXPIRE_FAST_CYCLE_DURATION</span></span><br><span class="line"><span class="comment"> * microseconds, and is not repeated again before the same amount of time.</span></span><br><span class="line"><span class="comment"> * The cycle will also refuse to run at all if the latest slow cycle did not</span></span><br><span class="line"><span class="comment"> * terminate because of a time limit condition.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * If type is ACTIVE_EXPIRE_CYCLE_SLOW, that normal expire cycle is</span></span><br><span class="line"><span class="comment"> * executed, where the time limit is a percentage of the REDIS_HZ period</span></span><br><span class="line"><span class="comment"> * as specified by the ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC define. In the</span></span><br><span class="line"><span class="comment"> * fast cycle, the check of every database is interrupted once the number</span></span><br><span class="line"><span class="comment"> * of already expired keys in the database is estimated to be lower than</span></span><br><span class="line"><span class="comment"> * a given percentage, in order to avoid doing too much work to gain too</span></span><br><span class="line"><span class="comment"> * little memory.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * The configured expire "effort" will modify the baseline parameters in</span></span><br><span class="line"><span class="comment"> * order to do more work in both the fast and slow expire cycles.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> ACTIVE_EXPIRE_CYCLE_KEYS_PER_LOOP 20 <span class="comment">/* Keys for each DB loop. */</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> ACTIVE_EXPIRE_CYCLE_FAST_DURATION 1000 <span class="comment">/* Microseconds. */</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC 25 <span class="comment">/* Max % of CPU to use. */</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> ACTIVE_EXPIRE_CYCLE_ACCEPTABLE_STALE 10 <span class="comment">/* % of stale keys after which</span></span></span><br><span class="line"><span class="comment"><span class="meta"> we do extra efforts. */</span></span></span><br><span class="line"><span class="type">void</span> <span class="title function_">activeExpireCycle</span><span class="params">(<span class="type">int</span> type)</span> {</span><br><span class="line"> <span class="comment">/* Adjust the running parameters according to the configured expire</span></span><br><span class="line"><span class="comment"> * effort. The default effort is 1, and the maximum configurable effort</span></span><br><span class="line"><span class="comment"> * is 10. */</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">long</span></span><br><span class="line"> effort = server.active_expire_effort<span class="number">-1</span>, <span class="comment">/* Rescale from 0 to 9. */</span></span><br><span class="line"> config_keys_per_loop = ACTIVE_EXPIRE_CYCLE_KEYS_PER_LOOP +</span><br><span class="line"> ACTIVE_EXPIRE_CYCLE_KEYS_PER_LOOP/<span class="number">4</span>*effort,</span><br><span class="line"> config_cycle_fast_duration = ACTIVE_EXPIRE_CYCLE_FAST_DURATION +</span><br><span class="line"> ACTIVE_EXPIRE_CYCLE_FAST_DURATION/<span class="number">4</span>*effort,</span><br><span class="line"> config_cycle_slow_time_perc = ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC +</span><br><span class="line"> <span class="number">2</span>*effort,</span><br><span class="line"> config_cycle_acceptable_stale = ACTIVE_EXPIRE_CYCLE_ACCEPTABLE_STALE-</span><br><span class="line"> effort;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* This function has some global state in order to continue the work</span></span><br><span class="line"><span class="comment"> * incrementally across calls. */</span></span><br><span class="line"> <span class="type">static</span> <span class="type">unsigned</span> <span class="type">int</span> current_db = <span class="number">0</span>; <span class="comment">/* Last DB tested. */</span></span><br><span class="line"> <span class="type">static</span> <span class="type">int</span> timelimit_exit = <span class="number">0</span>; <span class="comment">/* Time limit hit in previous call? */</span></span><br><span class="line"> <span class="type">static</span> <span class="type">long</span> <span class="type">long</span> last_fast_cycle = <span class="number">0</span>; <span class="comment">/* When last fast cycle ran. */</span></span><br><span class="line"></span><br><span class="line"> <span class="type">int</span> j, iteration = <span class="number">0</span>;</span><br><span class="line"> <span class="type">int</span> dbs_per_call = CRON_DBS_PER_CALL;</span><br><span class="line"> <span class="type">long</span> <span class="type">long</span> start = ustime(), timelimit, elapsed;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* When clients are paused the dataset should be static not just from the</span></span><br><span class="line"><span class="comment"> * POV of clients not being able to write, but also from the POV of</span></span><br><span class="line"><span class="comment"> * expires and evictions of keys not being performed. */</span></span><br><span class="line"> <span class="keyword">if</span> (clientsArePaused()) <span class="keyword">return</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (type == ACTIVE_EXPIRE_CYCLE_FAST) {</span><br><span class="line"> <span class="comment">/* Don't start a fast cycle if the previous cycle did not exit</span></span><br><span class="line"><span class="comment"> * for time limit, unless the percentage of estimated stale keys is</span></span><br><span class="line"><span class="comment"> * too high. Also never repeat a fast cycle for the same period</span></span><br><span class="line"><span class="comment"> * as the fast cycle total duration itself. */</span></span><br><span class="line"> <span class="keyword">if</span> (!timelimit_exit &&</span><br><span class="line"> server.stat_expired_stale_perc < config_cycle_acceptable_stale)</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (start < last_fast_cycle + (<span class="type">long</span> <span class="type">long</span>)config_cycle_fast_duration*<span class="number">2</span>)</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"></span><br><span class="line"> last_fast_cycle = start;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* We usually should test CRON_DBS_PER_CALL per iteration, with</span></span><br><span class="line"><span class="comment"> * two exceptions:</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * 1) Don't test more DBs than we have.</span></span><br><span class="line"><span class="comment"> * 2) If last time we hit the time limit, we want to scan all DBs</span></span><br><span class="line"><span class="comment"> * in this iteration, as there is work to do in some DB and we don't want</span></span><br><span class="line"><span class="comment"> * expired keys to use memory for too much time. */</span></span><br><span class="line"> <span class="keyword">if</span> (dbs_per_call > server.dbnum || timelimit_exit)</span><br><span class="line"> dbs_per_call = server.dbnum;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* We can use at max 'config_cycle_slow_time_perc' percentage of CPU</span></span><br><span class="line"><span class="comment"> * time per iteration. Since this function gets called with a frequency of</span></span><br><span class="line"><span class="comment"> * server.hz times per second, the following is the max amount of</span></span><br><span class="line"><span class="comment"> * microseconds we can spend in this function. */</span></span><br><span class="line"> timelimit = config_cycle_slow_time_perc*<span class="number">1000000</span>/server.hz/<span class="number">100</span>;</span><br><span class="line"> timelimit_exit = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">if</span> (timelimit <= <span class="number">0</span>) timelimit = <span class="number">1</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (type == ACTIVE_EXPIRE_CYCLE_FAST)</span><br><span class="line"> timelimit = config_cycle_fast_duration; <span class="comment">/* in microseconds. */</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Accumulate some global stats as we expire keys, to have some idea</span></span><br><span class="line"><span class="comment"> * about the number of keys that are already logically expired, but still</span></span><br><span class="line"><span class="comment"> * existing inside the database. */</span></span><br><span class="line"> <span class="type">long</span> total_sampled = <span class="number">0</span>;</span><br><span class="line"> <span class="type">long</span> total_expired = <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (j = <span class="number">0</span>; j < dbs_per_call && timelimit_exit == <span class="number">0</span>; j++) {</span><br><span class="line"> <span class="comment">/* Expired and checked in a single loop. */</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">long</span> expired, sampled;</span><br><span class="line"></span><br><span class="line"> redisDb *db = server.db+(current_db % server.dbnum);</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Increment the DB now so we are sure if we run out of time</span></span><br><span class="line"><span class="comment"> * in the current DB we'll restart from the next. This allows to</span></span><br><span class="line"><span class="comment"> * distribute the time evenly across DBs. */</span></span><br><span class="line"> current_db++;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Continue to expire if at the end of the cycle more than 25%</span></span><br><span class="line"><span class="comment"> * of the keys were expired. */</span></span><br><span class="line"> <span class="keyword">do</span> {</span><br><span class="line"> <span class="type">unsigned</span> <span class="type">long</span> num, slots;</span><br><span class="line"> <span class="type">long</span> <span class="type">long</span> now, ttl_sum;</span><br><span class="line"> <span class="type">int</span> ttl_samples;</span><br><span class="line"> iteration++;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* If there is nothing to expire try next DB ASAP. */</span></span><br><span class="line"> <span class="keyword">if</span> ((num = dictSize(db->expires)) == <span class="number">0</span>) {</span><br><span class="line"> db->avg_ttl = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> slots = dictSlots(db->expires);</span><br><span class="line"> now = mstime();</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* When there are less than 1% filled slots, sampling the key</span></span><br><span class="line"><span class="comment"> * space is expensive, so stop here waiting for better times...</span></span><br><span class="line"><span class="comment"> * The dictionary will be resized asap. */</span></span><br><span class="line"> <span class="keyword">if</span> (num && slots > DICT_HT_INITIAL_SIZE &&</span><br><span class="line"> (num*<span class="number">100</span>/slots < <span class="number">1</span>)) <span class="keyword">break</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* The main collection cycle. Sample random keys among keys</span></span><br><span class="line"><span class="comment"> * with an expire set, checking for expired ones. */</span></span><br><span class="line"> expired = <span class="number">0</span>;</span><br><span class="line"> sampled = <span class="number">0</span>;</span><br><span class="line"> ttl_sum = <span class="number">0</span>;</span><br><span class="line"> ttl_samples = <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (num > config_keys_per_loop)</span><br><span class="line"> num = config_keys_per_loop;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Here we access the low level representation of the hash table</span></span><br><span class="line"><span class="comment"> * for speed concerns: this makes this code coupled with dict.c,</span></span><br><span class="line"><span class="comment"> * but it hardly changed in ten years.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * Note that certain places of the hash table may be empty,</span></span><br><span class="line"><span class="comment"> * so we want also a stop condition about the number of</span></span><br><span class="line"><span class="comment"> * buckets that we scanned. However scanning for free buckets</span></span><br><span class="line"><span class="comment"> * is very fast: we are in the cache line scanning a sequential</span></span><br><span class="line"><span class="comment"> * array of NULL pointers, so we can scan a lot more buckets</span></span><br><span class="line"><span class="comment"> * than keys in the same time. */</span></span><br><span class="line"> <span class="type">long</span> max_buckets = num*<span class="number">20</span>;</span><br><span class="line"> <span class="type">long</span> checked_buckets = <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">while</span> (sampled < num && checked_buckets < max_buckets) {</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> table = <span class="number">0</span>; table < <span class="number">2</span>; table++) {</span><br><span class="line"> <span class="keyword">if</span> (table == <span class="number">1</span> && !dictIsRehashing(db->expires)) <span class="keyword">break</span>;</span><br><span class="line"></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">long</span> idx = db->expires_cursor;</span><br><span class="line"> idx &= db->expires->ht[table].sizemask;</span><br><span class="line"> dictEntry *de = db->expires->ht[table].table[idx];</span><br><span class="line"> <span class="type">long</span> <span class="type">long</span> ttl;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Scan the current bucket of the current table. */</span></span><br><span class="line"> checked_buckets++;</span><br><span class="line"> <span class="keyword">while</span>(de) {</span><br><span class="line"> <span class="comment">/* Get the next entry now since this entry may get</span></span><br><span class="line"><span class="comment"> * deleted. */</span></span><br><span class="line"> dictEntry *e = de;</span><br><span class="line"> de = de->next;</span><br><span class="line"></span><br><span class="line"> ttl = dictGetSignedIntegerVal(e)-now;</span><br><span class="line"> <span class="keyword">if</span> (activeExpireCycleTryExpire(db,e,now)) expired++;</span><br><span class="line"> <span class="keyword">if</span> (ttl > <span class="number">0</span>) {</span><br><span class="line"> <span class="comment">/* We want the average TTL of keys yet</span></span><br><span class="line"><span class="comment"> * not expired. */</span></span><br><span class="line"> ttl_sum += ttl;</span><br><span class="line"> ttl_samples++;</span><br><span class="line"> }</span><br><span class="line"> sampled++;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> db->expires_cursor++;</span><br><span class="line"> }</span><br><span class="line"> total_expired += expired;</span><br><span class="line"> total_sampled += sampled;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Update the average TTL stats for this database. */</span></span><br><span class="line"> <span class="keyword">if</span> (ttl_samples) {</span><br><span class="line"> <span class="type">long</span> <span class="type">long</span> avg_ttl = ttl_sum/ttl_samples;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Do a simple running average with a few samples.</span></span><br><span class="line"><span class="comment"> * We just use the current estimate with a weight of 2%</span></span><br><span class="line"><span class="comment"> * and the previous estimate with a weight of 98%. */</span></span><br><span class="line"> <span class="keyword">if</span> (db->avg_ttl == <span class="number">0</span>) db->avg_ttl = avg_ttl;</span><br><span class="line"> db->avg_ttl = (db->avg_ttl/<span class="number">50</span>)*<span class="number">49</span> + (avg_ttl/<span class="number">50</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* We can't block forever here even if there are many keys to</span></span><br><span class="line"><span class="comment"> * expire. So after a given amount of milliseconds return to the</span></span><br><span class="line"><span class="comment"> * caller waiting for the other active expire cycle. */</span></span><br><span class="line"> <span class="keyword">if</span> ((iteration & <span class="number">0xf</span>) == <span class="number">0</span>) { <span class="comment">/* check once every 16 iterations. */</span></span><br><span class="line"> elapsed = ustime()-start;</span><br><span class="line"> <span class="keyword">if</span> (elapsed > timelimit) {</span><br><span class="line"> timelimit_exit = <span class="number">1</span>;</span><br><span class="line"> server.stat_expired_time_cap_reached_count++;</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">/* We don't repeat the cycle for the current database if there are</span></span><br><span class="line"><span class="comment"> * an acceptable amount of stale keys (logically expired but yet</span></span><br><span class="line"><span class="comment"> * not reclained). */</span></span><br><span class="line"> } <span class="keyword">while</span> ((expired*<span class="number">100</span>/sampled) > config_cycle_acceptable_stale);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> elapsed = ustime()-start;</span><br><span class="line"> server.stat_expire_cycle_time_used += elapsed;</span><br><span class="line"> latencyAddSampleIfNeeded(<span class="string">"expire-cycle"</span>,elapsed/<span class="number">1000</span>);</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Update our estimate of keys existing but yet to be expired.</span></span><br><span class="line"><span class="comment"> * Running average with this sample accounting for 5%. */</span></span><br><span class="line"> <span class="type">double</span> current_perc;</span><br><span class="line"> <span class="keyword">if</span> (total_sampled) {</span><br><span class="line"> current_perc = (<span class="type">double</span>)total_expired/total_sampled;</span><br><span class="line"> } <span class="keyword">else</span></span><br><span class="line"> current_perc = <span class="number">0</span>;</span><br><span class="line"> server.stat_expired_stale_perc = (current_perc*<span class="number">0.05</span>)+</span><br><span class="line"> (server.stat_expired_stale_perc*<span class="number">0.95</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>执行定期清除分成两种类型，快和慢，分别由<code>beforeSleep</code>和<code>databasesCron</code>调用，快版有两个限制，一个是执行时长由ACTIVE_EXPIRE_CYCLE_FAST_DURATION限制，另一个是执行间隔是 2 倍的ACTIVE_EXPIRE_CYCLE_FAST_DURATION，另外这还可以由配置的server.active_expire_effort参数来控制，默认是 1，最大是 10</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line">onfig_cycle_fast_duration = ACTIVE_EXPIRE_CYCLE_FAST_DURATION +</span><br><span class="line"> ACTIVE_EXPIRE_CYCLE_FAST_DURATION/<span class="number">4</span>*effort</span><br></pre></td></tr></table></figure>
|
|
<p>然后会从一定数量的 db 中找出一定数量的带过期时间的 key（保存在 expires中），这里的数量是由</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line">config_keys_per_loop = ACTIVE_EXPIRE_CYCLE_KEYS_PER_LOOP +</span><br><span class="line"> ACTIVE_EXPIRE_CYCLE_KEYS_PER_LOOP/<span class="number">4</span>*effort</span><br><span class="line">``` </span><br><span class="line">控制，慢速的执行时长是</span><br><span class="line">```C</span><br><span class="line">config_cycle_slow_time_perc = ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC +</span><br><span class="line"> <span class="number">2</span>*effort</span><br><span class="line">timelimit = config_cycle_slow_time_perc*<span class="number">1000000</span>/server.hz/<span class="number">100</span>;</span><br></pre></td></tr></table></figure>
|
|
<p>这里还有一个额外的退出条件，如果当前数据库的抽样结果已经达到我们所允许的过期 key 百分比，则下次不再处理当前 db，继续处理下个 db</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Redis</category>
|
|
<category>数据结构</category>
|
|
<category>C</category>
|
|
<category>源码</category>
|
|
<category>Redis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>redis</tag>
|
|
<tag>数据结构</tag>
|
|
<tag>源码</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>redis系列介绍八-淘汰策略</title>
|
|
<url>/2020/04/18/redis%E7%B3%BB%E5%88%97%E4%BB%8B%E7%BB%8D%E5%85%AB/</url>
|
|
<content><![CDATA[<h3 id="LRU"><a href="#LRU" class="headerlink" title="LRU"></a>LRU</h3><p>说完了过期策略再说下淘汰策略，redis 使用的策略是近似的 lru 策略，为什么是近似的呢，先来看下什么是 lru，看下 wiki 的介绍<br><img data-src="https://img.nicksxs.me/uPic/z5cDpo.jpg">，图中一共有四个槽的存储空间，依次访问顺序是 A B C D E D F，<br>当第一次访问 D 时刚好占满了坑，并且值是 4，这个值越小代表越先被淘汰，当 E 进来时，看了下已经存在的四个里 A 是最小的，代表是最早存在并且最早被访问的，那就先淘汰它了，E 占领了 A 的位置，并设置值为 4，然后又访问 D 了，D 已经存在了，不过又被访问到了，得更新值为 5，然后是 F 进来了，这时 B 是最老的且最近未被访问，所以就淘汰它了。以上是一个 lru 的简要说明，但是 redis 没有严格按照这个去执行，理由跟前面过期策略一致，最严格的过期策略应该是每个 key 都有对应的定时器，当超时时马上就能清除，但是问题是这样的cpu 消耗太大，所换来的内存效率不太值得，淘汰策略也是这样，类似于上图，要维护所有 key 的一个有序 lru 值，并且遍历将最小的淘汰，redis 采用的是抽样的形式，最初的实现方式是随机从 dict 抽取 5 个 key，淘汰一个 lru 最小的，这样子勉强能达到淘汰的目的，但是效果不是特别好，后面在 redis 3.0开始，将随机抽取改成了维护一个 pool，pool 的大小默认是 16，每次放入的都是按lru 值有序排列好，每一次放入的必须是 lru小于 pool 中最小的 lru 才允许放入，直到放满，后面再有新的就会将大的踢出。<br>redis 针对这个策略的改进做了一个实验，这里借用下图<br><img data-src="https://img.nicksxs.me/uPic/lEx4Ug.jpg"><br>首先背景是这图中的所有点都对应一个 redis 的 key，灰色部分加入后被顺序访问过一遍，然后又加入了绿色部分，那么按照理论的 lru 算法，应该是图左上中，浅灰色部分全都被淘汰，那么对比来看看图右上，左下和右下，左下表示 2.8 版本就是随机抽样 5 个 key，淘汰其中 lru 最小的一个，发现是灰色和浅灰色的都有被淘汰的，右下的 3.0 版本抽样数量不变的情况下，稍好一些，当 3.0 版本的抽样数量调整成 10 后，已经较为接近理论上的 lru 策略了，通过代码来简要分析下</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">redisObject</span> {</span></span><br><span class="line"> <span class="type">unsigned</span> type:<span class="number">4</span>;</span><br><span class="line"> <span class="type">unsigned</span> encoding:<span class="number">4</span>;</span><br><span class="line"> <span class="type">unsigned</span> lru:LRU_BITS; <span class="comment">/* LRU time (relative to global lru_clock) or</span></span><br><span class="line"><span class="comment"> * LFU data (least significant 8 bits frequency</span></span><br><span class="line"><span class="comment"> * and most significant 16 bits access time). */</span></span><br><span class="line"> <span class="type">int</span> refcount;</span><br><span class="line"> <span class="type">void</span> *ptr;</span><br><span class="line">} robj;</span><br></pre></td></tr></table></figure>
|
|
<p>对于 lru 策略来说，lru 字段记录的就是<a href="https://github.com/antirez/redis/blob/unstable/src/server.h#L603"><code>redisObj</code></a> 的LRU time，<br>redis 在访问数据时，都会调用<a href="https://github.com/antirez/redis/blob/unstable/src/db.c#L55"><code>lookupKey</code></a>方法</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="comment">/* Low level key lookup API, not actually called directly from commands</span></span><br><span class="line"><span class="comment"> * implementations that should instead rely on lookupKeyRead(),</span></span><br><span class="line"><span class="comment"> * lookupKeyWrite() and lookupKeyReadWithFlags(). */</span></span><br><span class="line">robj *<span class="title function_">lookupKey</span><span class="params">(redisDb *db, robj *key, <span class="type">int</span> flags)</span> {</span><br><span class="line"> dictEntry *de = dictFind(db->dict,key->ptr);</span><br><span class="line"> <span class="keyword">if</span> (de) {</span><br><span class="line"> robj *val = dictGetVal(de);</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Update the access time for the ageing algorithm.</span></span><br><span class="line"><span class="comment"> * Don't do it if we have a saving child, as this will trigger</span></span><br><span class="line"><span class="comment"> * a copy on write madness. */</span></span><br><span class="line"> <span class="keyword">if</span> (!hasActiveChildProcess() && !(flags & LOOKUP_NOTOUCH)){</span><br><span class="line"> <span class="keyword">if</span> (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {</span><br><span class="line"> <span class="comment">// 这个是后面一节的内容</span></span><br><span class="line"> updateLFU(val);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// 对于这个分支，访问时就会去更新 lru 值</span></span><br><span class="line"> val->lru = LRU_CLOCK();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> val;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">NULL</span>;</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"><span class="comment">/* This function is used to obtain the current LRU clock.</span></span><br><span class="line"><span class="comment"> * If the current resolution is lower than the frequency we refresh the</span></span><br><span class="line"><span class="comment"> * LRU clock (as it should be in production servers) we return the</span></span><br><span class="line"><span class="comment"> * precomputed value, otherwise we need to resort to a system call. */</span></span><br><span class="line"><span class="type">unsigned</span> <span class="type">int</span> <span class="title function_">LRU_CLOCK</span><span class="params">(<span class="type">void</span>)</span> {</span><br><span class="line"> <span class="type">unsigned</span> <span class="type">int</span> lruclock;</span><br><span class="line"> <span class="keyword">if</span> (<span class="number">1000</span>/server.hz <= LRU_CLOCK_RESOLUTION) {</span><br><span class="line"> <span class="comment">// 如果服务器的频率server.hz大于 1 时就是用系统预设的 lruclock</span></span><br><span class="line"> lruclock = server.lruclock;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> lruclock = getLRUClock();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> lruclock;</span><br><span class="line">}</span><br><span class="line"><span class="comment">/* Return the LRU clock, based on the clock resolution. This is a time</span></span><br><span class="line"><span class="comment"> * in a reduced-bits format that can be used to set and check the</span></span><br><span class="line"><span class="comment"> * object->lru field of redisObject structures. */</span></span><br><span class="line"><span class="type">unsigned</span> <span class="type">int</span> <span class="title function_">getLRUClock</span><span class="params">(<span class="type">void</span>)</span> {</span><br><span class="line"> <span class="keyword">return</span> (mstime()/LRU_CLOCK_RESOLUTION) & LRU_CLOCK_MAX;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>redis 处理命令是在这里<a href="https://github.com/antirez/redis/blob/unstable/src/server.c#L3355"><code>processCommand</code></a></p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="comment">/* If this function gets called we already read a whole</span></span><br><span class="line"><span class="comment"> * command, arguments are in the client argv/argc fields.</span></span><br><span class="line"><span class="comment"> * processCommand() execute the command or prepare the</span></span><br><span class="line"><span class="comment"> * server for a bulk read from the client.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * If C_OK is returned the client is still alive and valid and</span></span><br><span class="line"><span class="comment"> * other operations can be performed by the caller. Otherwise</span></span><br><span class="line"><span class="comment"> * if C_ERR is returned the client was destroyed (i.e. after QUIT). */</span></span><br><span class="line"><span class="type">int</span> <span class="title function_">processCommand</span><span class="params">(client *c)</span> {</span><br><span class="line"> moduleCallCommandFilters(c);</span><br><span class="line"></span><br><span class="line"> </span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Handle the maxmemory directive.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * Note that we do not want to reclaim memory if we are here re-entering</span></span><br><span class="line"><span class="comment"> * the event loop since there is a busy Lua script running in timeout</span></span><br><span class="line"><span class="comment"> * condition, to avoid mixing the propagation of scripts with the</span></span><br><span class="line"><span class="comment"> * propagation of DELs due to eviction. */</span></span><br><span class="line"> <span class="keyword">if</span> (server.maxmemory && !server.lua_timedout) {</span><br><span class="line"> <span class="type">int</span> out_of_memory = freeMemoryIfNeededAndSafe() == C_ERR;</span><br><span class="line"> <span class="comment">/* freeMemoryIfNeeded may flush slave output buffers. This may result</span></span><br><span class="line"><span class="comment"> * into a slave, that may be the active client, to be freed. */</span></span><br><span class="line"> <span class="keyword">if</span> (server.current_client == <span class="literal">NULL</span>) <span class="keyword">return</span> C_ERR;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* It was impossible to free enough memory, and the command the client</span></span><br><span class="line"><span class="comment"> * is trying to execute is denied during OOM conditions or the client</span></span><br><span class="line"><span class="comment"> * is in MULTI/EXEC context? Error. */</span></span><br><span class="line"> <span class="keyword">if</span> (out_of_memory &&</span><br><span class="line"> (c->cmd->flags & CMD_DENYOOM ||</span><br><span class="line"> (c->flags & CLIENT_MULTI &&</span><br><span class="line"> c->cmd->proc != execCommand &&</span><br><span class="line"> c->cmd->proc != discardCommand)))</span><br><span class="line"> {</span><br><span class="line"> flagTransaction(c);</span><br><span class="line"> addReply(c, shared.oomerr);</span><br><span class="line"> <span class="keyword">return</span> C_OK;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里只摘了部分，当需要清理内存时就会调用, 然后调用了<a href="https://github.com/antirez/redis/blob/unstable/src/evict.c#L631"><code>freeMemoryIfNeededAndSafe</code></a></p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="comment">/* This is a wrapper for freeMemoryIfNeeded() that only really calls the</span></span><br><span class="line"><span class="comment"> * function if right now there are the conditions to do so safely:</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * - There must be no script in timeout condition.</span></span><br><span class="line"><span class="comment"> * - Nor we are loading data right now.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="type">int</span> <span class="title function_">freeMemoryIfNeededAndSafe</span><span class="params">(<span class="type">void</span>)</span> {</span><br><span class="line"> <span class="keyword">if</span> (server.lua_timedout || server.loading) <span class="keyword">return</span> C_OK;</span><br><span class="line"> <span class="keyword">return</span> freeMemoryIfNeeded();</span><br><span class="line">}</span><br><span class="line"><span class="comment">/* This function is periodically called to see if there is memory to free</span></span><br><span class="line"><span class="comment"> * according to the current "maxmemory" settings. In case we are over the</span></span><br><span class="line"><span class="comment"> * memory limit, the function will try to free some memory to return back</span></span><br><span class="line"><span class="comment"> * under the limit.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * The function returns C_OK if we are under the memory limit or if we</span></span><br><span class="line"><span class="comment"> * were over the limit, but the attempt to free memory was successful.</span></span><br><span class="line"><span class="comment"> * Otehrwise if we are over the memory limit, but not enough memory</span></span><br><span class="line"><span class="comment"> * was freed to return back under the limit, the function returns C_ERR. */</span></span><br><span class="line"><span class="type">int</span> <span class="title function_">freeMemoryIfNeeded</span><span class="params">(<span class="type">void</span>)</span> {</span><br><span class="line"> <span class="type">int</span> keys_freed = <span class="number">0</span>;</span><br><span class="line"> <span class="comment">/* By default replicas should ignore maxmemory</span></span><br><span class="line"><span class="comment"> * and just be masters exact copies. */</span></span><br><span class="line"> <span class="keyword">if</span> (server.masterhost && server.repl_slave_ignore_maxmemory) <span class="keyword">return</span> C_OK;</span><br><span class="line"></span><br><span class="line"> <span class="type">size_t</span> mem_reported, mem_tofree, mem_freed;</span><br><span class="line"> <span class="type">mstime_t</span> latency, eviction_latency;</span><br><span class="line"> <span class="type">long</span> <span class="type">long</span> delta;</span><br><span class="line"> <span class="type">int</span> slaves = listLength(server.slaves);</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* When clients are paused the dataset should be static not just from the</span></span><br><span class="line"><span class="comment"> * POV of clients not being able to write, but also from the POV of</span></span><br><span class="line"><span class="comment"> * expires and evictions of keys not being performed. */</span></span><br><span class="line"> <span class="keyword">if</span> (clientsArePaused()) <span class="keyword">return</span> C_OK;</span><br><span class="line"> <span class="keyword">if</span> (getMaxmemoryState(&mem_reported,<span class="literal">NULL</span>,&mem_tofree,<span class="literal">NULL</span>) == C_OK)</span><br><span class="line"> <span class="keyword">return</span> C_OK;</span><br><span class="line"></span><br><span class="line"> mem_freed = <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (server.maxmemory_policy == MAXMEMORY_NO_EVICTION)</span><br><span class="line"> <span class="keyword">goto</span> cant_free; <span class="comment">/* We need to free memory, but policy forbids. */</span></span><br><span class="line"></span><br><span class="line"> latencyStartMonitor(latency);</span><br><span class="line"> <span class="keyword">while</span> (mem_freed < mem_tofree) {</span><br><span class="line"> <span class="type">int</span> j, k, i;</span><br><span class="line"> <span class="type">static</span> <span class="type">unsigned</span> <span class="type">int</span> next_db = <span class="number">0</span>;</span><br><span class="line"> sds bestkey = <span class="literal">NULL</span>;</span><br><span class="line"> <span class="type">int</span> bestdbid;</span><br><span class="line"> redisDb *db;</span><br><span class="line"> dict *dict;</span><br><span class="line"> dictEntry *de;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (server.maxmemory_policy & (MAXMEMORY_FLAG_LRU|MAXMEMORY_FLAG_LFU) ||</span><br><span class="line"> server.maxmemory_policy == MAXMEMORY_VOLATILE_TTL)</span><br><span class="line"> {</span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">evictionPoolEntry</span> *<span class="title">pool</span> =</span> EvictionPoolLRU;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">while</span>(bestkey == <span class="literal">NULL</span>) {</span><br><span class="line"> <span class="type">unsigned</span> <span class="type">long</span> total_keys = <span class="number">0</span>, keys;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* We don't want to make local-db choices when expiring keys,</span></span><br><span class="line"><span class="comment"> * so to start populate the eviction pool sampling keys from</span></span><br><span class="line"><span class="comment"> * every DB. */</span></span><br><span class="line"> <span class="keyword">for</span> (i = <span class="number">0</span>; i < server.dbnum; i++) {</span><br><span class="line"> db = server.db+i;</span><br><span class="line"> dict = (server.maxmemory_policy & MAXMEMORY_FLAG_ALLKEYS) ?</span><br><span class="line"> db->dict : db->expires;</span><br><span class="line"> <span class="keyword">if</span> ((keys = dictSize(dict)) != <span class="number">0</span>) {</span><br><span class="line"> evictionPoolPopulate(i, dict, db->dict, pool);</span><br><span class="line"> total_keys += keys;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (!total_keys) <span class="keyword">break</span>; <span class="comment">/* No keys to evict. */</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Go backward from best to worst element to evict. */</span></span><br><span class="line"> <span class="keyword">for</span> (k = EVPOOL_SIZE<span class="number">-1</span>; k >= <span class="number">0</span>; k--) {</span><br><span class="line"> <span class="keyword">if</span> (pool[k].key == <span class="literal">NULL</span>) <span class="keyword">continue</span>;</span><br><span class="line"> bestdbid = pool[k].dbid;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (server.maxmemory_policy & MAXMEMORY_FLAG_ALLKEYS) {</span><br><span class="line"> de = dictFind(server.db[pool[k].dbid].dict,</span><br><span class="line"> pool[k].key);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> de = dictFind(server.db[pool[k].dbid].expires,</span><br><span class="line"> pool[k].key);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Remove the entry from the pool. */</span></span><br><span class="line"> <span class="keyword">if</span> (pool[k].key != pool[k].cached)</span><br><span class="line"> sdsfree(pool[k].key);</span><br><span class="line"> pool[k].key = <span class="literal">NULL</span>;</span><br><span class="line"> pool[k].idle = <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* If the key exists, is our pick. Otherwise it is</span></span><br><span class="line"><span class="comment"> * a ghost and we need to try the next element. */</span></span><br><span class="line"> <span class="keyword">if</span> (de) {</span><br><span class="line"> bestkey = dictGetKey(de);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">/* Ghost... Iterate again. */</span></span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* volatile-random and allkeys-random policy */</span></span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span> (server.maxmemory_policy == MAXMEMORY_ALLKEYS_RANDOM ||</span><br><span class="line"> server.maxmemory_policy == MAXMEMORY_VOLATILE_RANDOM)</span><br><span class="line"> {</span><br><span class="line"> <span class="comment">/* When evicting a random key, we try to evict a key for</span></span><br><span class="line"><span class="comment"> * each DB, so we use the static 'next_db' variable to</span></span><br><span class="line"><span class="comment"> * incrementally visit all DBs. */</span></span><br><span class="line"> <span class="keyword">for</span> (i = <span class="number">0</span>; i < server.dbnum; i++) {</span><br><span class="line"> j = (++next_db) % server.dbnum;</span><br><span class="line"> db = server.db+j;</span><br><span class="line"> dict = (server.maxmemory_policy == MAXMEMORY_ALLKEYS_RANDOM) ?</span><br><span class="line"> db->dict : db->expires;</span><br><span class="line"> <span class="keyword">if</span> (dictSize(dict) != <span class="number">0</span>) {</span><br><span class="line"> de = dictGetRandomKey(dict);</span><br><span class="line"> bestkey = dictGetKey(de);</span><br><span class="line"> bestdbid = j;</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Finally remove the selected key. */</span></span><br><span class="line"> <span class="keyword">if</span> (bestkey) {</span><br><span class="line"> db = server.db+bestdbid;</span><br><span class="line"> robj *keyobj = createStringObject(bestkey,sdslen(bestkey));</span><br><span class="line"> propagateExpire(db,keyobj,server.lazyfree_lazy_eviction);</span><br><span class="line"> <span class="comment">/* We compute the amount of memory freed by db*Delete() alone.</span></span><br><span class="line"><span class="comment"> * It is possible that actually the memory needed to propagate</span></span><br><span class="line"><span class="comment"> * the DEL in AOF and replication link is greater than the one</span></span><br><span class="line"><span class="comment"> * we are freeing removing the key, but we can't account for</span></span><br><span class="line"><span class="comment"> * that otherwise we would never exit the loop.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * AOF and Output buffer memory will be freed eventually so</span></span><br><span class="line"><span class="comment"> * we only care about memory used by the key space. */</span></span><br><span class="line"> delta = (<span class="type">long</span> <span class="type">long</span>) zmalloc_used_memory();</span><br><span class="line"> latencyStartMonitor(eviction_latency);</span><br><span class="line"> <span class="keyword">if</span> (server.lazyfree_lazy_eviction)</span><br><span class="line"> dbAsyncDelete(db,keyobj);</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> dbSyncDelete(db,keyobj);</span><br><span class="line"> latencyEndMonitor(eviction_latency);</span><br><span class="line"> latencyAddSampleIfNeeded(<span class="string">"eviction-del"</span>,eviction_latency);</span><br><span class="line"> latencyRemoveNestedEvent(latency,eviction_latency);</span><br><span class="line"> delta -= (<span class="type">long</span> <span class="type">long</span>) zmalloc_used_memory();</span><br><span class="line"> mem_freed += delta;</span><br><span class="line"> server.stat_evictedkeys++;</span><br><span class="line"> notifyKeyspaceEvent(NOTIFY_EVICTED, <span class="string">"evicted"</span>,</span><br><span class="line"> keyobj, db->id);</span><br><span class="line"> decrRefCount(keyobj);</span><br><span class="line"> keys_freed++;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* When the memory to free starts to be big enough, we may</span></span><br><span class="line"><span class="comment"> * start spending so much time here that is impossible to</span></span><br><span class="line"><span class="comment"> * deliver data to the slaves fast enough, so we force the</span></span><br><span class="line"><span class="comment"> * transmission here inside the loop. */</span></span><br><span class="line"> <span class="keyword">if</span> (slaves) flushSlavesOutputBuffers();</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Normally our stop condition is the ability to release</span></span><br><span class="line"><span class="comment"> * a fixed, pre-computed amount of memory. However when we</span></span><br><span class="line"><span class="comment"> * are deleting objects in another thread, it's better to</span></span><br><span class="line"><span class="comment"> * check, from time to time, if we already reached our target</span></span><br><span class="line"><span class="comment"> * memory, since the "mem_freed" amount is computed only</span></span><br><span class="line"><span class="comment"> * across the dbAsyncDelete() call, while the thread can</span></span><br><span class="line"><span class="comment"> * release the memory all the time. */</span></span><br><span class="line"> <span class="keyword">if</span> (server.lazyfree_lazy_eviction && !(keys_freed % <span class="number">16</span>)) {</span><br><span class="line"> <span class="keyword">if</span> (getMaxmemoryState(<span class="literal">NULL</span>,<span class="literal">NULL</span>,<span class="literal">NULL</span>,<span class="literal">NULL</span>) == C_OK) {</span><br><span class="line"> <span class="comment">/* Let's satisfy our stop condition. */</span></span><br><span class="line"> mem_freed = mem_tofree;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> latencyEndMonitor(latency);</span><br><span class="line"> latencyAddSampleIfNeeded(<span class="string">"eviction-cycle"</span>,latency);</span><br><span class="line"> <span class="keyword">goto</span> cant_free; <span class="comment">/* nothing to free... */</span></span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> latencyEndMonitor(latency);</span><br><span class="line"> latencyAddSampleIfNeeded(<span class="string">"eviction-cycle"</span>,latency);</span><br><span class="line"> <span class="keyword">return</span> C_OK;</span><br><span class="line"></span><br><span class="line">cant_free:</span><br><span class="line"> <span class="comment">/* We are here if we are not able to reclaim memory. There is only one</span></span><br><span class="line"><span class="comment"> * last thing we can try: check if the lazyfree thread has jobs in queue</span></span><br><span class="line"><span class="comment"> * and wait... */</span></span><br><span class="line"> <span class="keyword">while</span>(bioPendingJobsOfType(BIO_LAZY_FREE)) {</span><br><span class="line"> <span class="keyword">if</span> (((mem_reported - zmalloc_used_memory()) + mem_freed) >= mem_tofree)</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> usleep(<span class="number">1000</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> C_ERR;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里就是根据具体策略去淘汰 key，首先是要往 pool 更新 key，更新key 的方法是<a href="https://github.com/antirez/redis/blob/unstable/src/evict.c#L162"><code>evictionPoolPopulate</code></a></p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="type">void</span> <span class="title function_">evictionPoolPopulate</span><span class="params">(<span class="type">int</span> dbid, dict *sampledict, dict *keydict, <span class="keyword">struct</span> evictionPoolEntry *pool)</span> {</span><br><span class="line"> <span class="type">int</span> j, k, count;</span><br><span class="line"> dictEntry *samples[server.maxmemory_samples];</span><br><span class="line"></span><br><span class="line"> count = dictGetSomeKeys(sampledict,samples,server.maxmemory_samples);</span><br><span class="line"> <span class="keyword">for</span> (j = <span class="number">0</span>; j < count; j++) {</span><br><span class="line"> <span class="type">unsigned</span> <span class="type">long</span> <span class="type">long</span> idle;</span><br><span class="line"> sds key;</span><br><span class="line"> robj *o;</span><br><span class="line"> dictEntry *de;</span><br><span class="line"></span><br><span class="line"> de = samples[j];</span><br><span class="line"> key = dictGetKey(de);</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* If the dictionary we are sampling from is not the main</span></span><br><span class="line"><span class="comment"> * dictionary (but the expires one) we need to lookup the key</span></span><br><span class="line"><span class="comment"> * again in the key dictionary to obtain the value object. */</span></span><br><span class="line"> <span class="keyword">if</span> (server.maxmemory_policy != MAXMEMORY_VOLATILE_TTL) {</span><br><span class="line"> <span class="keyword">if</span> (sampledict != keydict) de = dictFind(keydict, key);</span><br><span class="line"> o = dictGetVal(de);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Calculate the idle time according to the policy. This is called</span></span><br><span class="line"><span class="comment"> * idle just because the code initially handled LRU, but is in fact</span></span><br><span class="line"><span class="comment"> * just a score where an higher score means better candidate. */</span></span><br><span class="line"> <span class="keyword">if</span> (server.maxmemory_policy & MAXMEMORY_FLAG_LRU) {</span><br><span class="line"> idle = estimateObjectIdleTime(o);</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {</span><br><span class="line"> <span class="comment">/* When we use an LRU policy, we sort the keys by idle time</span></span><br><span class="line"><span class="comment"> * so that we expire keys starting from greater idle time.</span></span><br><span class="line"><span class="comment"> * However when the policy is an LFU one, we have a frequency</span></span><br><span class="line"><span class="comment"> * estimation, and we want to evict keys with lower frequency</span></span><br><span class="line"><span class="comment"> * first. So inside the pool we put objects using the inverted</span></span><br><span class="line"><span class="comment"> * frequency subtracting the actual frequency to the maximum</span></span><br><span class="line"><span class="comment"> * frequency of 255. */</span></span><br><span class="line"> idle = <span class="number">255</span>-LFUDecrAndReturn(o);</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (server.maxmemory_policy == MAXMEMORY_VOLATILE_TTL) {</span><br><span class="line"> <span class="comment">/* In this case the sooner the expire the better. */</span></span><br><span class="line"> idle = ULLONG_MAX - (<span class="type">long</span>)dictGetVal(de);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> serverPanic(<span class="string">"Unknown eviction policy in evictionPoolPopulate()"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Insert the element inside the pool.</span></span><br><span class="line"><span class="comment"> * First, find the first empty bucket or the first populated</span></span><br><span class="line"><span class="comment"> * bucket that has an idle time smaller than our idle time. */</span></span><br><span class="line"> k = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">while</span> (k < EVPOOL_SIZE &&</span><br><span class="line"> pool[k].key &&</span><br><span class="line"> pool[k].idle < idle) k++;</span><br><span class="line"> <span class="keyword">if</span> (k == <span class="number">0</span> && pool[EVPOOL_SIZE<span class="number">-1</span>].key != <span class="literal">NULL</span>) {</span><br><span class="line"> <span class="comment">/* Can't insert if the element is < the worst element we have</span></span><br><span class="line"><span class="comment"> * and there are no empty buckets. */</span></span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (k < EVPOOL_SIZE && pool[k].key == <span class="literal">NULL</span>) {</span><br><span class="line"> <span class="comment">/* Inserting into empty position. No setup needed before insert. */</span></span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">/* Inserting in the middle. Now k points to the first element</span></span><br><span class="line"><span class="comment"> * greater than the element to insert. */</span></span><br><span class="line"> <span class="keyword">if</span> (pool[EVPOOL_SIZE<span class="number">-1</span>].key == <span class="literal">NULL</span>) {</span><br><span class="line"> <span class="comment">/* Free space on the right? Insert at k shifting</span></span><br><span class="line"><span class="comment"> * all the elements from k to end to the right. */</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Save SDS before overwriting. */</span></span><br><span class="line"> sds cached = pool[EVPOOL_SIZE<span class="number">-1</span>].cached;</span><br><span class="line"> memmove(pool+k+<span class="number">1</span>,pool+k,</span><br><span class="line"> <span class="keyword">sizeof</span>(pool[<span class="number">0</span>])*(EVPOOL_SIZE-k<span class="number">-1</span>));</span><br><span class="line"> pool[k].cached = cached;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">/* No free space on right? Insert at k-1 */</span></span><br><span class="line"> k--;</span><br><span class="line"> <span class="comment">/* Shift all elements on the left of k (included) to the</span></span><br><span class="line"><span class="comment"> * left, so we discard the element with smaller idle time. */</span></span><br><span class="line"> sds cached = pool[<span class="number">0</span>].cached; <span class="comment">/* Save SDS before overwriting. */</span></span><br><span class="line"> <span class="keyword">if</span> (pool[<span class="number">0</span>].key != pool[<span class="number">0</span>].cached) sdsfree(pool[<span class="number">0</span>].key);</span><br><span class="line"> memmove(pool,pool+<span class="number">1</span>,<span class="keyword">sizeof</span>(pool[<span class="number">0</span>])*k);</span><br><span class="line"> pool[k].cached = cached;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/* Try to reuse the cached SDS string allocated in the pool entry,</span></span><br><span class="line"><span class="comment"> * because allocating and deallocating this object is costly</span></span><br><span class="line"><span class="comment"> * (according to the profiler, not my fantasy. Remember:</span></span><br><span class="line"><span class="comment"> * premature optimizbla bla bla bla. */</span></span><br><span class="line"> <span class="type">int</span> klen = sdslen(key);</span><br><span class="line"> <span class="keyword">if</span> (klen > EVPOOL_CACHED_SDS_SIZE) {</span><br><span class="line"> pool[k].key = sdsdup(key);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="built_in">memcpy</span>(pool[k].cached,key,klen+<span class="number">1</span>);</span><br><span class="line"> sdssetlen(pool[k].cached,klen);</span><br><span class="line"> pool[k].key = pool[k].cached;</span><br><span class="line"> }</span><br><span class="line"> pool[k].idle = idle;</span><br><span class="line"> pool[k].dbid = dbid;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p><code>Redis</code>随机选择<code>maxmemory_samples</code>数量的key，然后计算这些<code>key</code>的空闲时间<code>idle time</code>，当满足条件时(比pool中的某些键的空闲时间还大)就可以进<code>pool</code>。<code>pool</code>更新之后，就淘汰<code>pool</code>中空闲时间最大的键。</p>
|
|
<p><a href="https://github.com/antirez/redis/blob/unstable/src/evict.c#L90"><code>estimateObjectIdleTime</code></a>用来计算Redis对象的空闲时间：</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="comment">/* Given an object returns the min number of milliseconds the object was never</span></span><br><span class="line"><span class="comment"> * requested, using an approximated LRU algorithm. */</span></span><br><span class="line"><span class="type">unsigned</span> <span class="type">long</span> <span class="type">long</span> <span class="title function_">estimateObjectIdleTime</span><span class="params">(robj *o)</span> {</span><br><span class="line"> <span class="type">unsigned</span> <span class="type">long</span> <span class="type">long</span> lruclock = LRU_CLOCK();</span><br><span class="line"> <span class="keyword">if</span> (lruclock >= o->lru) {</span><br><span class="line"> <span class="keyword">return</span> (lruclock - o->lru) * LRU_CLOCK_RESOLUTION;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> (lruclock + (LRU_CLOCK_MAX - o->lru)) *</span><br><span class="line"> LRU_CLOCK_RESOLUTION;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>空闲时间第一种是 lurclock 大于对象的 lru，那么就是减一下乘以精度，因为 lruclock 有可能是已经预生成的，所以会可能走下面这个</p>
|
|
<h3 id="LFU"><a href="#LFU" class="headerlink" title="LFU"></a>LFU</h3><p>上面介绍了LRU 的算法，但是考虑一种场景</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">~~~~~A~~~~~A~~~~~A~~~~A~~~~~A~~~~~A~~|</span><br><span class="line">~~B~~B~~B~~B~~B~~B~~B~~B~~B~~B~~B~~B~|</span><br><span class="line">~~~~~~~~~~C~~~~~~~~~C~~~~~~~~~C~~~~~~|</span><br><span class="line">~~~~~D~~~~~~~~~~D~~~~~~~~~D~~~~~~~~~D|</span><br></pre></td></tr></table></figure>
|
|
<p>可以发现，当采用 lru 的淘汰策略的时候，D 是最新的，会被认为是最值得保留的，但是事实上还不如 A 跟 B，然后 antirez 大神就想到了LFU (Least Frequently Used) 这个算法, 显然对于上面的四个 key 的访问频率，保留优先级应该是 B > A > C = D<br>那要怎么来实现这个 LFU 算法呢，其实像LRU，理想的情况就是维护个链表，把最新访问的放到头上去，但是这个会影响访问速度，注意到前面代码的应该可以看到，redisObject 的 lru 字段其实是两用的，当策略是 LFU 时，这个字段就另作他用了，它的 24 位长度被分成两部分</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line"> 16 bits 8 bits</span><br><span class="line">+----------------+--------+</span><br><span class="line">+ Last decr time | LOG_C |</span><br><span class="line">+----------------+--------+</span><br></pre></td></tr></table></figure>
|
|
<p>前16位字段是最后一次递减时间，因此Redis知道 上一次计数器递减，后8位是 计数器 counter。<br>LFU 的主体策略就是当这个 key 被访问的次数越多频率越高他就越容易被保留下来，并且是最近被访问的频率越高。这其实有两个事情要做，一个是在访问的时候增加计数值，在一定长时间不访问时进行衰减，所以这里用了两个值，前 16 位记录上一次衰减的时间，后 8 位记录具体的计数值。<br>Redis4.0之后为maxmemory_policy淘汰策略添加了两个LFU模式：</p>
|
|
<p><code>volatile-lfu</code>：对有过期时间的key采用LFU淘汰策略<br><code>allkeys-lfu</code>：对全部key采用LFU淘汰策略<br>还有2个配置可以调整LFU算法： </p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">lfu-log-factor 10</span><br><span class="line">lfu-decay-time 1</span><br><span class="line">``` </span><br><span class="line">`lfu-log-factor` 可以调整计数器counter的增长速度，lfu-log-factor越大，counter增长的越慢。</span><br><span class="line"></span><br><span class="line">`lfu-decay-time`是一个以分钟为单位的数值，可以调整counter的减少速度</span><br><span class="line">这里有个问题是 8 位大小够计么，访问一次加 1 的话的确不够，不过大神就是大神，才不会这么简单的加一。往下看代码</span><br><span class="line">```C</span><br><span class="line">/* Low level key lookup API, not actually called directly from commands</span><br><span class="line"> * implementations that should instead rely on lookupKeyRead(),</span><br><span class="line"> * lookupKeyWrite() and lookupKeyReadWithFlags(). */</span><br><span class="line">robj *lookupKey(redisDb *db, robj *key, int flags) {</span><br><span class="line"> dictEntry *de = dictFind(db->dict,key->ptr);</span><br><span class="line"> if (de) {</span><br><span class="line"> robj *val = dictGetVal(de);</span><br><span class="line"></span><br><span class="line"> /* Update the access time for the ageing algorithm.</span><br><span class="line"> * Don't do it if we have a saving child, as this will trigger</span><br><span class="line"> * a copy on write madness. */</span><br><span class="line"> if (!hasActiveChildProcess() && !(flags & LOOKUP_NOTOUCH)){</span><br><span class="line"> if (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {</span><br><span class="line"> // 当淘汰策略是 LFU 时，就会调用这个updateLFU</span><br><span class="line"> updateLFU(val);</span><br><span class="line"> } else {</span><br><span class="line"> val->lru = LRU_CLOCK();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> return val;</span><br><span class="line"> } else {</span><br><span class="line"> return NULL;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p><a href="https://github.com/antirez/redis/blob/unstable/src/db.c#L46"><code>updateLFU</code></a> 这个其实个入口，调用了两个重要的方法</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="comment">/* Update LFU when an object is accessed.</span></span><br><span class="line"><span class="comment"> * Firstly, decrement the counter if the decrement time is reached.</span></span><br><span class="line"><span class="comment"> * Then logarithmically increment the counter, and update the access time. */</span></span><br><span class="line"><span class="type">void</span> <span class="title function_">updateLFU</span><span class="params">(robj *val)</span> {</span><br><span class="line"> <span class="type">unsigned</span> <span class="type">long</span> counter = LFUDecrAndReturn(val);</span><br><span class="line"> counter = LFULogIncr(counter);</span><br><span class="line"> val->lru = (LFUGetTimeInMinutes()<<<span class="number">8</span>) | counter;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>首先来看看<a href="https://github.com/antirez/redis/blob/unstable/src/evict.c#L335"><code>LFUDecrAndReturn</code></a>，这个方法的作用是根据上一次衰减时间和系统配置的 <code>lfu-decay-time</code> 参数来确定需要将 counter 减去多少</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="comment">/* If the object decrement time is reached decrement the LFU counter but</span></span><br><span class="line"><span class="comment"> * do not update LFU fields of the object, we update the access time</span></span><br><span class="line"><span class="comment"> * and counter in an explicit way when the object is really accessed.</span></span><br><span class="line"><span class="comment"> * And we will times halve the counter according to the times of</span></span><br><span class="line"><span class="comment"> * elapsed time than server.lfu_decay_time.</span></span><br><span class="line"><span class="comment"> * Return the object frequency counter.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * This function is used in order to scan the dataset for the best object</span></span><br><span class="line"><span class="comment"> * to fit: as we check for the candidate, we incrementally decrement the</span></span><br><span class="line"><span class="comment"> * counter of the scanned objects if needed. */</span></span><br><span class="line"><span class="type">unsigned</span> <span class="type">long</span> <span class="title function_">LFUDecrAndReturn</span><span class="params">(robj *o)</span> {</span><br><span class="line"> <span class="comment">// 右移 8 位，拿到上次衰减时间</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">long</span> ldt = o->lru >> <span class="number">8</span>;</span><br><span class="line"> <span class="comment">// 对 255 做与操作，拿到 counter 值</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">long</span> counter = o->lru & <span class="number">255</span>;</span><br><span class="line"> <span class="comment">// 根据lfu_decay_time来算出过了多少个衰减周期</span></span><br><span class="line"> <span class="type">unsigned</span> <span class="type">long</span> num_periods = server.lfu_decay_time ? LFUTimeElapsed(ldt) / server.lfu_decay_time : <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">if</span> (num_periods)</span><br><span class="line"> counter = (num_periods > counter) ? <span class="number">0</span> : counter - num_periods;</span><br><span class="line"> <span class="keyword">return</span> counter;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后是加，调用了<a href="https://github.com/antirez/redis/blob/unstable/src/evict.c#L315"><code>LFULogIncr</code></a></p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="comment">/* Logarithmically increment a counter. The greater is the current counter value</span></span><br><span class="line"><span class="comment"> * the less likely is that it gets really implemented. Saturate it at 255. */</span></span><br><span class="line"><span class="type">uint8_t</span> <span class="title function_">LFULogIncr</span><span class="params">(<span class="type">uint8_t</span> counter)</span> {</span><br><span class="line"> <span class="comment">// 最大值就是 255，到顶了就不加了</span></span><br><span class="line"> <span class="keyword">if</span> (counter == <span class="number">255</span>) <span class="keyword">return</span> <span class="number">255</span>;</span><br><span class="line"> <span class="comment">// 生成个随机小数</span></span><br><span class="line"> <span class="type">double</span> r = (<span class="type">double</span>)rand()/RAND_MAX;</span><br><span class="line"> <span class="comment">// 减去个基础值，LFU_INIT_VAL = 5，防止刚进来就被逐出</span></span><br><span class="line"> <span class="type">double</span> baseval = counter - LFU_INIT_VAL;</span><br><span class="line"> <span class="comment">// 如果是小于 0，</span></span><br><span class="line"> <span class="keyword">if</span> (baseval < <span class="number">0</span>) baseval = <span class="number">0</span>;</span><br><span class="line"> <span class="comment">// 如果 baseval 是 0，那么 p 就是 1了，后面 counter 直接加一，如果不是的话，得看系统参数lfu_log_factor，这个越大，除出来的 p 越小，那么 counter++的可能性也越小，这样子就把前面的疑问给解决了，不是直接+1 的</span></span><br><span class="line"> <span class="type">double</span> p = <span class="number">1.0</span>/(baseval*server.lfu_log_factor+<span class="number">1</span>);</span><br><span class="line"> <span class="keyword">if</span> (r < p) counter++;</span><br><span class="line"> <span class="keyword">return</span> counter;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>大概的变化速度可以参考</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">+--------+------------+------------+------------+------------+------------+</span><br><span class="line">| factor | 100 hits | 1000 hits | 100K hits | 1M hits | 10M hits |</span><br><span class="line">+--------+------------+------------+------------+------------+------------+</span><br><span class="line">| 0 | 104 | 255 | 255 | 255 | 255 |</span><br><span class="line">+--------+------------+------------+------------+------------+------------+</span><br><span class="line">| 1 | 18 | 49 | 255 | 255 | 255 |</span><br><span class="line">+--------+------------+------------+------------+------------+------------+</span><br><span class="line">| 10 | 10 | 18 | 142 | 255 | 255 |</span><br><span class="line">+--------+------------+------------+------------+------------+------------+</span><br><span class="line">| 100 | 8 | 11 | 49 | 143 | 255 |</span><br><span class="line">+--------+------------+------------+------------+------------+------------+</span><br></pre></td></tr></table></figure>
|
|
<p>简而言之就是 lfu_log_factor 越大变化的越慢</p>
|
|
<h3 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h3><p>总结一下，redis 实现了近似的 lru 淘汰策略，通过增加了淘汰 key 的池子(pool)，并且增大每次抽样的 key 的数量来将淘汰效果更进一步地接近于 lru，这是 lru 策略，但是对于前面举的一个例子，其实 lru 并不能保证 key 的淘汰就如我们预期，所以在后期又引入了 lfu 的策略，lfu的策略比较巧妙，复用了 redis 对象的 lru 字段，并且使用了factor 参数来控制计数器递增的速度，防止 8 位的计数器太早溢出。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Redis</category>
|
|
<category>数据结构</category>
|
|
<category>C</category>
|
|
<category>源码</category>
|
|
<category>Redis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>redis</tag>
|
|
<tag>数据结构</tag>
|
|
<tag>源码</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>java的字节码工具-javassist体验二</title>
|
|
<url>/2025/01/12/java%E7%9A%84%E5%AD%97%E8%8A%82%E7%A0%81%E5%B7%A5%E5%85%B7-javassist%E4%BD%93%E9%AA%8C%E4%BA%8C/</url>
|
|
<content><![CDATA[<p>上次说了可以改写类，那进一步的我们可以做一下类似于之前提过的通过字节码来做切面的工作<br>首先我们有一个很简单的类和方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">DemoService</span> {</span><br><span class="line"> <span class="keyword">public</span> List<String> <span class="title function_">queryList</span><span class="params">(String name, String title, String n3)</span> {</span><br><span class="line"> List<String> list = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"> list.add(name);</span><br><span class="line"> list.add(title);</span><br><span class="line"> <span class="keyword">return</span> list;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>就是一个方法，输入几个字符串，然后加到list里，那么比如我们想知道打印入参和返回结果<br>这里比较类似于我们在用aop的时候就是织入代码<br>正好javassist有这样的操作接口<br>就是我们可以找到方法，然后使用</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">javassist.CtBehavior#insertBefore(java.lang.String)</span><br></pre></td></tr></table></figure>
|
|
<p>和</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">javassist.CtBehavior#insertAfter(java.lang.String)</span><br></pre></td></tr></table></figure>
|
|
<p>就可以在我们找到类和方法以后</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">ClassPool</span> <span class="variable">pool</span> <span class="operator">=</span> ClassPool.getDefault();</span><br><span class="line"><span class="type">CtClass</span> <span class="variable">cc</span> <span class="operator">=</span> pool.get(targetClass);</span><br><span class="line"><span class="type">CtMethod</span> <span class="variable">method</span> <span class="operator">=</span> cc.getDeclaredMethod(targetMethod);</span><br></pre></td></tr></table></figure>
|
|
<p>对方法的前后进行逻辑织入</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">method.insertBefore(</span><br><span class="line"> <span class="string">"{ java.lang.StringBuilder _paramValues = new StringBuilder();"</span> +</span><br><span class="line"> <span class="string">" Object[] params = $args;"</span> +</span><br><span class="line"> <span class="string">" if(params != null) {"</span> +</span><br><span class="line"> <span class="string">" for(int i=0; i<params.length; i++) {"</span> +</span><br><span class="line"> <span class="string">" if(i > 0) _paramValues.append(\", \");"</span> +</span><br><span class="line"> <span class="string">" _paramValues.append(String.valueOf(params[i]));"</span> +</span><br><span class="line"> <span class="string">" }"</span> +</span><br><span class="line"> <span class="string">" }"</span> +</span><br><span class="line"> <span class="string">" System.out.println(\"[Method Entry] "</span> + targetMethod + <span class="string">" params: \" + _paramValues.toString());"</span> +</span><br><span class="line"> <span class="string">"}"</span></span><br><span class="line"> );</span><br></pre></td></tr></table></figure>
|
|
<p>这里我们使用了直接织入代码的形式，在javassist中可以用 <code>$args</code> 来获取参数，然后循环参数, 把每个参数拼接以后作为字符串输出<br>然后我们想要在函数返回前获取返回值，这个也是有个语法糖 <code>$_</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">StringBuilder</span> <span class="variable">after</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">StringBuilder</span>();</span><br><span class="line"> after.append(<span class="string">"{"</span>)</span><br><span class="line"> .append(<span class="string">" java.util.List _result = $_;\n"</span>)</span><br><span class="line"> .append(<span class="string">" String _resultStr = (_result != null ? _result.toString() : \"null\");\n"</span>)</span><br><span class="line"> .append(<span class="string">" int _size = (_result != null ? _result.size() : 0);\n"</span>)</span><br><span class="line"> .append(<span class="string">" System.out.println(\"[Method Exit] "</span> + targetMethod + <span class="string">" returns: \" + _resultStr + \" size: \" + _size);\n"</span>)</span><br><span class="line"> .append(<span class="string">"}"</span>);</span><br></pre></td></tr></table></figure>
|
|
<p>这里刚好我是list返回的，就直接这么处理了, 我们通过一个简单的调用来验证下</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="comment">// 假设要监控 org.example.DemoService 类中的 queryList 方法</span></span><br><span class="line"> addMonitoringNew(<span class="string">"org.example.DemoService"</span>, <span class="string">"queryList"</span>);</span><br><span class="line"> <span class="comment">// 2. 使用自定义类加载器加载增强后的类</span></span><br><span class="line"> <span class="type">ClassLoader</span> <span class="variable">classLoader</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ClassLoader</span>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> Class<?> loadClass(String name) <span class="keyword">throws</span> ClassNotFoundException {</span><br><span class="line"> <span class="keyword">if</span> (name.equals(<span class="string">"org.example.DemoService"</span>)) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 读取增强后的类文件</span></span><br><span class="line"> <span class="type">byte</span>[] bytes = java.nio.file.Files.readAllBytes(</span><br><span class="line"> java.nio.file.Paths.get(<span class="string">"org/example/DemoService.class"</span>)</span><br><span class="line"> );</span><br><span class="line"> <span class="keyword">return</span> defineClass(name, bytes, <span class="number">0</span>, bytes.length);</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">ClassNotFoundException</span>(<span class="string">"Failed to load enhanced class"</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">super</span>.loadClass(name);</span><br><span class="line"> }</span><br><span class="line"> };</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 3. 加载增强后的类</span></span><br><span class="line"> Class<?> enhancedClass = classLoader.loadClass(<span class="string">"org.example.DemoService"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 4. 创建实例</span></span><br><span class="line"> <span class="type">Object</span> <span class="variable">instance</span> <span class="operator">=</span> enhancedClass.newInstance();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 5. 获取并调用方法</span></span><br><span class="line"> <span class="type">Method</span> <span class="variable">processMethod</span> <span class="operator">=</span> enhancedClass.getMethod(<span class="string">"queryList"</span>, String.class, String.class, String.class); <span class="comment">// 假设方法接收一个String参数</span></span><br><span class="line"> <span class="type">Object</span> <span class="variable">result</span> <span class="operator">=</span> processMethod.invoke(instance, <span class="string">"name1"</span>, <span class="string">"title1"</span>, <span class="string">"n3"</span>);</span><br><span class="line"></span><br><span class="line"> System.out.println(<span class="string">"Method returned: "</span> + result);</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line">可以看到我们的输出结果</span><br><span class="line">```java</span><br><span class="line">Successfully enhanced method: queryList</span><br><span class="line">[Method Entry] queryList params: name1, title1, n3</span><br><span class="line">[Method Exit] queryList returns: [name1, title1] size: <span class="number">2</span></span><br><span class="line">Method returned: [name1, title1]</span><br></pre></td></tr></table></figure>
|
|
<p>这里其实核心很依赖两个语法糖，类似这样的还有一些</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">$<span class="number">0</span>, $<span class="number">1</span>, $<span class="number">2</span>, ... <span class="built_in">this</span> and actual parameters</span><br><span class="line">$args An array of parameters. The type of $args is Object[].</span><br><span class="line">$$ All actual parameters.</span><br><span class="line">For example, m($$) is equivalent to <span class="title function_">m</span><span class="params">($<span class="number">1</span>,$<span class="number">2</span>,...)</span></span><br><span class="line"> </span><br><span class="line">$cflow(...) cflow variable</span><br><span class="line">$r The result type. It is used in a cast expression.</span><br><span class="line">$w The wrapper type. It is used in a cast expression.</span><br><span class="line">$_ The resulting value</span><br><span class="line">$sig An array of java.lang.Class objects representing the formal parameter types.</span><br><span class="line">$type A java.lang.Class object representing the formal result type.</span><br><span class="line">$<span class="keyword">class</span> <span class="title class_">A</span> java.lang.Class object representing the <span class="keyword">class</span> <span class="title class_">currently</span> edited.</span><br></pre></td></tr></table></figure>
|
|
<p>可以结合<a href="https://www.javassist.org/tutorial/tutorial2.html">文档</a>仔细学习下</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>redis过期策略复习</title>
|
|
<url>/2021/07/25/redis%E8%BF%87%E6%9C%9F%E7%AD%96%E7%95%A5%E5%A4%8D%E4%B9%A0/</url>
|
|
<content><![CDATA[<h1 id="redis过期策略复习"><a href="#redis过期策略复习" class="headerlink" title="redis过期策略复习"></a>redis过期策略复习</h1><p>之前其实写过redis的过期的一些原理，这次主要是记录下，一些使用上的概念，主要是redis使用的过期策略是懒过期和定时清除，懒过期的其实比较简单，即是在key被访问的时候会顺带着判断下这个key是否已过期了，如果已经过期了，就不返回了，但是这种策略有个漏洞是如果有些key之后一直不会被访问了，就等于沉在池底了，所以需要有一个定时的清理机制，去从设置了过期的key池子（expires）里随机地捞key，具体的策略我们看下官网的解释</p>
|
|
<ol>
|
|
<li>Test 20 random keys from the set of keys with an associated expire.</li>
|
|
<li>Delete all the keys found expired.</li>
|
|
<li>If more than 25% of keys were expired, start again from step 1.</li>
|
|
</ol>
|
|
<p>从池子里随机获取20个key，将其中过期的key删掉，如果这其中有超过25%的key已经过期了，那就再来一次，以此保持过期的key不超过25%（左右），并且这个定时策略可以在redis的配置文件</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_"># </span><span class="language-bash">Redis calls an internal <span class="keyword">function</span> to perform many background tasks, like</span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">closing connections of clients <span class="keyword">in</span> <span class="built_in">timeout</span>, purging expired keys that are</span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">never requested, and so forth.</span></span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash"></span></span><br><span class="line"><span class="language-bash"><span class="comment"># Not all tasks are performed with the same frequency, but Redis checks for</span></span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">tasks to perform according to the specified <span class="string">"hz"</span> value.</span></span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash"></span></span><br><span class="line"><span class="language-bash"><span class="comment"># By default "hz" is set to 10. Raising the value will use more CPU when</span></span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">Redis is idle, but at the same time will make Redis more responsive when</span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">there are many keys expiring at the same time, and timeouts may be</span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">handled with more precision.</span></span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash"></span></span><br><span class="line"><span class="language-bash"><span class="comment"># The range is between 1 and 500, however a value over 100 is usually not</span></span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">a good idea. Most <span class="built_in">users</span> should use the default of 10 and raise this up to</span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">100 only <span class="keyword">in</span> environments <span class="built_in">where</span> very low latency is required.</span></span><br><span class="line">hz 10</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>可以配置这个hz的值，代表的含义是每秒的执行次数，默认是10，其实也用了hz的普遍含义。有兴趣可以看看之前写的一篇文章<a href="%5Bhttps://nicksxs.me/2020/04/12/redis%E7%B3%BB%E5%88%97%E4%BB%8B%E7%BB%8D%E4%B8%83/%5D(https://nicksxs.me/2020/04/12/redis%E7%B3%BB%E5%88%97%E4%BB%8B%E7%BB%8D%E4%B8%83/)">redis系列介绍七-过期策略</a></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>redis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>redis</tag>
|
|
<tag>应用</tag>
|
|
<tag>过期策略</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>rust学习笔记-所有权三之切片</title>
|
|
<url>/2021/05/16/rust%E5%AD%A6%E4%B9%A0%E7%AC%94%E8%AE%B0-%E6%89%80%E6%9C%89%E6%9D%83%E4%B8%89%E4%B9%8B%E5%88%87%E7%89%87/</url>
|
|
<content><![CDATA[<p>除了引用，Rust 还有另外一种不持有所有权的数据类型：切片(slice)。切片允许我们引用集合中某一段连续的元素序列，而不是整个集合。<br>例如代码</p>
|
|
<figure class="highlight rust"><table><tr><td class="code"><pre><span class="line"><span class="keyword">fn</span> <span class="title function_">main</span>() {</span><br><span class="line"> <span class="keyword">let</span> <span class="keyword">mut </span><span class="variable">s</span> = <span class="type">String</span>::<span class="title function_ invoke__">from</span>(<span class="string">"hello world"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">let</span> <span class="variable">word</span> = <span class="title function_ invoke__">first_word</span>(&s);</span><br><span class="line"></span><br><span class="line"> s.<span class="title function_ invoke__">clear</span>();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 这时候虽然 word 还是 5，但是 s 已经被清除了，所以就没存在的意义</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里其实我们就需要关注 s 的存在性，代码的逻辑合理性就需要额外去维护，此时我们就可以用切片</p>
|
|
<figure class="highlight rust"><table><tr><td class="code"><pre><span class="line"><span class="keyword">let</span> <span class="variable">s</span> = <span class="type">String</span>::<span class="title function_ invoke__">from</span>(<span class="string">"hello world"</span>)</span><br><span class="line"></span><br><span class="line"><span class="keyword">let</span> <span class="variable">hello</span> = &s[<span class="number">0</span>..<span class="number">5</span>];</span><br><span class="line"><span class="keyword">let</span> <span class="variable">world</span> = &s[<span class="number">6</span>..<span class="number">11</span>];</span><br></pre></td></tr></table></figure>
|
|
<p>其实跟 Python 的list 之类的语法有点类似，当然里面还有些语法糖，比如可以直接用省略后面的数字表示直接引用到结尾</p>
|
|
<figure class="highlight rust"><table><tr><td class="code"><pre><span class="line"><span class="keyword">let</span> <span class="variable">hello</span> = &s[<span class="number">0</span>..];</span><br></pre></td></tr></table></figure>
|
|
<p>甚至再进一步</p>
|
|
<figure class="highlight rust"><table><tr><td class="code"><pre><span class="line"><span class="keyword">let</span> <span class="variable">hello</span> = &s[..];</span><br></pre></td></tr></table></figure>
|
|
<p>使用了切片之后</p>
|
|
<figure class="highlight rust"><table><tr><td class="code"><pre><span class="line"><span class="keyword">fn</span> <span class="title function_">first_word</span>(s: &<span class="type">String</span>) <span class="punctuation">-></span> &<span class="type">str</span> {</span><br><span class="line"> <span class="keyword">let</span> <span class="variable">bytes</span> = s.<span class="title function_ invoke__">as_bytes</span>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (i, &item) <span class="keyword">in</span> bytes.<span class="title function_ invoke__">iter</span>().<span class="title function_ invoke__">enumerate</span>() {</span><br><span class="line"> <span class="keyword">if</span> item == <span class="string">b' '</span> {</span><br><span class="line"> <span class="keyword">return</span> &s[<span class="number">0</span>..i];</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> &s[..]</span><br><span class="line">}</span><br><span class="line"><span class="keyword">fn</span> <span class="title function_">main</span>() {</span><br><span class="line"> <span class="keyword">let</span> <span class="keyword">mut </span><span class="variable">s</span> = <span class="type">String</span>::<span class="title function_ invoke__">from</span>(<span class="string">"hello world"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">let</span> <span class="variable">word</span> = <span class="title function_ invoke__">first_word</span>(&s);</span><br><span class="line"></span><br><span class="line"> s.<span class="title function_ invoke__">clear</span>(); <span class="comment">// error!</span></span><br><span class="line"></span><br><span class="line"> <span class="built_in">println!</span>(<span class="string">"the first word is: {}"</span>, word);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>那再执行 main 函数的时候就会抛错，因为 word 还是个切片，需要保证 s 的有效性，并且其实我们可以将函数申明成</p>
|
|
<figure class="highlight rust"><table><tr><td class="code"><pre><span class="line"><span class="keyword">fn</span> <span class="title function_">first_word</span>(s: &<span class="type">str</span>) <span class="punctuation">-></span> &<span class="type">str</span> {</span><br></pre></td></tr></table></figure>
|
|
<p>这样就既能处理&String 的情况，就是当成完整字符串的切片，也能处理普通的切片。<br>其他类型的切片</p>
|
|
<figure class="highlight rust"><table><tr><td class="code"><pre><span class="line"><span class="keyword">let</span> <span class="variable">a</span> = [<span class="number">1</span>, <span class="number">2</span>, <span class="number">3</span>, <span class="number">4</span>, <span class="number">5</span>];</span><br><span class="line"><span class="keyword">let</span> <span class="variable">slice</span> = &a[<span class="number">1</span>..<span class="number">3</span>];</span><br></pre></td></tr></table></figure>
|
|
<p>简单记录下，具体可以去看看这本书</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>语言</category>
|
|
<category>Rust</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Rust</tag>
|
|
<tag>所有权</tag>
|
|
<tag>内存分布</tag>
|
|
<tag>新语言</tag>
|
|
<tag>可变引用</tag>
|
|
<tag>不可变引用</tag>
|
|
<tag>切片</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>rust学习笔记-所有权一</title>
|
|
<url>/2021/04/18/rust%E5%AD%A6%E4%B9%A0%E7%AC%94%E8%AE%B0/</url>
|
|
<content><![CDATA[<p>最近在看 《rust 权威指南》，还是难度比较大的，它里面的一些概念跟之前的用过的都有比较大的差别<br>比起有 gc 的虚拟机语言，跟像 C 和 C++这种主动释放内存的，rust 有他的独特点，主要是有三条</p>
|
|
<ul>
|
|
<li>Rust中的每一个值都有一个对应的变量作为它的所有者。</li>
|
|
<li>在同一时间内，值有且只有一个所有者。</li>
|
|
<li>当所有者离开自己的作用域时，它持有的值就会被释放掉。<br><img data-src="https://img.nicksxs.me/uPic/U2pUVH.png"><br>这里有两个重点：</li>
|
|
<li>s 在进入作用域后才变得有效</li>
|
|
<li>它会保持自己的有效性直到自己离开作用域为止</li>
|
|
</ul>
|
|
<p>然后看个案例</p>
|
|
<figure class="highlight rust"><table><tr><td class="code"><pre><span class="line"><span class="keyword">let</span> <span class="variable">x</span> = <span class="number">5</span>;</span><br><span class="line"><span class="keyword">let</span> <span class="variable">y</span> = x;</span><br></pre></td></tr></table></figure>
|
|
<p>这个其实有两种，一般可以认为比较多实现的会使用 copy on write 之类的，先让两个都指向同一个快 5 的存储，在发生变更后开始正式拷贝，但是涉及到内存处理的便利性，对于这类简单类型，可以直接拷贝<br>但是对于非基础类型</p>
|
|
<figure class="highlight rust"><table><tr><td class="code"><pre><span class="line"><span class="keyword">let</span> <span class="variable">s1</span> = <span class="type">String</span>::<span class="title function_ invoke__">from</span>(<span class="string">"hello"</span>);</span><br><span class="line"><span class="keyword">let</span> <span class="variable">s2</span> = s1;</span><br><span class="line"></span><br><span class="line"><span class="built_in">println!</span>(<span class="string">"{}, world!"</span>, s1);</span><br></pre></td></tr></table></figure>
|
|
<p>有可能认为有两种内存分布可能<br>先看下 string 的内存结构<br><img data-src="https://img.nicksxs.me/uPic/mhfEjF.png"><br>第一种可能是<br><img data-src="https://img.nicksxs.me/uPic/dDTQTu.png"><br>第二种是<br><img data-src="https://img.nicksxs.me/uPic/ZNUCbc.png"><br>我们来尝试编译下<br><img data-src="https://img.nicksxs.me/uPic/X6teOn.png"><br>发现有这个错误，其实在 rust 中<code>let y = x</code>这个行为的实质是移动，在赋值给 y 之后 x 就无效了<br><img data-src="https://img.nicksxs.me/uPic/gfJcts.png"><br>这样子就不会造成脱离作用域时，对同一块内存区域的二次释放，如果需要复制，可以使用 clone 方法</p>
|
|
<figure class="highlight rust"><table><tr><td class="code"><pre><span class="line"><span class="keyword">let</span> <span class="variable">s1</span> = <span class="type">String</span>::<span class="title function_ invoke__">from</span>(<span class="string">"hello"</span>);</span><br><span class="line"><span class="keyword">let</span> <span class="variable">s2</span> = s1.<span class="title function_ invoke__">clone</span>();</span><br><span class="line"></span><br><span class="line"><span class="built_in">println!</span>(<span class="string">"s1 = {}, s2 = {}"</span>, s1, s2);</span><br></pre></td></tr></table></figure>
|
|
<p>这里其实会有点疑惑，为什么前面的<code>x, y</code> 的行为跟 <code>s1, s2</code> 的不一样，其实主要是基本类型和 string 这类的不定大小的类型的内存分配方式不同，<code>x, y</code>这类整型可以直接确定大小，可以直接在栈上分配，而像 string 和其他的变体结构体，其大小都是不能在编译时确定，所以需要在堆上进行分配</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>语言</category>
|
|
<category>Rust</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Rust</tag>
|
|
<tag>所有权</tag>
|
|
<tag>内存分布</tag>
|
|
<tag>新语言</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>rust学习笔记-所有权二</title>
|
|
<url>/2021/04/18/rust%E5%AD%A6%E4%B9%A0%E7%AC%94%E8%AE%B0-%E6%89%80%E6%9C%89%E6%9D%83%E4%BA%8C/</url>
|
|
<content><![CDATA[<p>这里需要说道函数和返回值了<br>可以看书上的这个例子<br><img data-src="https://img.nicksxs.me/uPic/fnlsui.png"><br>对于这种情况，当进入函数内部时，会把传入的变量的所有权转移进函数内部，如果最后还是要返回该变量，但是如果此时还要返回别的计算结果，就可能需要笨拙地使用元组<br><img data-src="https://img.nicksxs.me/uPic/6vBWwi.png"></p>
|
|
<h3 id="引用"><a href="#引用" class="headerlink" title="引用"></a>引用</h3><p>此时我们就可以用引用来解决这个问题</p>
|
|
<figure class="highlight rust"><table><tr><td class="code"><pre><span class="line"><span class="keyword">fn</span> <span class="title function_">main</span>() {</span><br><span class="line"> <span class="keyword">let</span> <span class="variable">s1</span> = <span class="type">String</span>::<span class="title function_ invoke__">from</span>(<span class="string">"hello"</span>);</span><br><span class="line"> <span class="keyword">let</span> <span class="variable">len</span> = <span class="title function_ invoke__">calculate_length</span>(&s1);</span><br><span class="line"></span><br><span class="line"> <span class="built_in">println!</span>(<span class="string">"The length of '{}' is {}"</span>, s1, len);</span><br><span class="line">}</span><br><span class="line"><span class="keyword">fn</span> <span class="title function_">calculate_length</span>(s: &<span class="type">String</span>) <span class="punctuation">-></span> <span class="type">usize</span> {</span><br><span class="line"> s.<span class="title function_ invoke__">len</span>()</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里的&符号就是引用的语义，它们允许你在不获得所有权的前提下使用值<br><img data-src="https://img.nicksxs.me/uPic/nniiIt.png"><br>由于引用不持有值的所有权，所以当引用离开当前作用域时，它指向的值也不会被丢弃</p>
|
|
<h3 id="可变引用"><a href="#可变引用" class="headerlink" title="可变引用"></a>可变引用</h3><p>而当我们尝试对引用的字符串进行修改时</p>
|
|
<figure class="highlight rust"><table><tr><td class="code"><pre><span class="line"><span class="keyword">fn</span> <span class="title function_">main</span>() {</span><br><span class="line"> <span class="keyword">let</span> <span class="variable">s1</span> = <span class="type">String</span>::<span class="title function_ invoke__">from</span>(<span class="string">"hello"</span>);</span><br><span class="line"> <span class="title function_ invoke__">change</span>(&s1);</span><br><span class="line">}</span><br><span class="line"><span class="keyword">fn</span> <span class="title function_">change</span>(s: &<span class="type">String</span>) {</span><br><span class="line"> s.<span class="title function_ invoke__">push_str</span>(<span class="string">", world"</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>就会有以下报错，<br><img data-src="https://img.nicksxs.me/uPic/LVWURi.png"><br>其实也很容易发现，毕竟没有 mut 指出这是可变引用，同时需要将 s1 改成 mut 可变的</p>
|
|
<figure class="highlight rust"><table><tr><td class="code"><pre><span class="line"><span class="keyword">fn</span> <span class="title function_">main</span>() {</span><br><span class="line"> <span class="keyword">let</span> <span class="keyword">mut </span><span class="variable">s1</span> = <span class="type">String</span>::<span class="title function_ invoke__">from</span>(<span class="string">"hello"</span>);</span><br><span class="line"> <span class="title function_ invoke__">change</span>(&<span class="keyword">mut</span> s1);</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">fn</span> <span class="title function_">change</span>(s: &<span class="keyword">mut</span> <span class="type">String</span>) {</span><br><span class="line"> s.<span class="title function_ invoke__">push_str</span>(<span class="string">", world"</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>再看一个例子</p>
|
|
<figure class="highlight rust"><table><tr><td class="code"><pre><span class="line"><span class="keyword">fn</span> <span class="title function_">main</span>() {</span><br><span class="line"> <span class="keyword">let</span> <span class="keyword">mut </span><span class="variable">s1</span> = <span class="type">String</span>::<span class="title function_ invoke__">from</span>(<span class="string">"hello"</span>);</span><br><span class="line"> <span class="keyword">let</span> <span class="variable">r1</span> = &<span class="keyword">mut</span> s1;</span><br><span class="line"> <span class="keyword">let</span> <span class="variable">r2</span> = &<span class="keyword">mut</span> s1;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这个例子在书里是会报错的，因为同时存在一个以上的可变引用，但是在我运行的版本里前面这段没有报错，只有当我真的要去更改的时候</p>
|
|
<figure class="highlight rust"><table><tr><td class="code"><pre><span class="line"><span class="keyword">fn</span> <span class="title function_">main</span>() {</span><br><span class="line"> <span class="keyword">let</span> <span class="keyword">mut </span><span class="variable">s1</span> = <span class="type">String</span>::<span class="title function_ invoke__">from</span>(<span class="string">"hello"</span>);</span><br><span class="line"> <span class="keyword">let</span> <span class="keyword">mut </span><span class="variable">r1</span> = &<span class="keyword">mut</span> s1;</span><br><span class="line"> <span class="keyword">let</span> <span class="keyword">mut </span><span class="variable">r2</span> = &<span class="keyword">mut</span> s1;</span><br><span class="line"> <span class="title function_ invoke__">change</span>(&<span class="keyword">mut</span> r1);</span><br><span class="line"> <span class="title function_ invoke__">change</span>(&<span class="keyword">mut</span> r2);</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">fn</span> <span class="title function_">change</span>(s: &<span class="keyword">mut</span> <span class="type">String</span>) {</span><br><span class="line"> s.<span class="title function_ invoke__">push_str</span>(<span class="string">", world"</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/uPic/k3jjNn.png"><br>这里可能就是具体版本在实现上的一个差异，我用的 rustc 是 1.44.0 版本<br>其实上面的主要是由 rust 想要避免这类多重可变更导致的异常问题，总结下就是三个点</p>
|
|
<ul>
|
|
<li>两个或两个以上的指针同时同时访问同一空间</li>
|
|
<li>其中至少有一个指针会想空间中写入数据</li>
|
|
<li>没有同步数据访问的机制<br>并且我们不能在拥有不可变引用的情况下创建可变引用</li>
|
|
</ul>
|
|
<h3 id="悬垂引用"><a href="#悬垂引用" class="headerlink" title="悬垂引用"></a>悬垂引用</h3><p>还有一点需要注意的就是悬垂引用</p>
|
|
<figure class="highlight rust"><table><tr><td class="code"><pre><span class="line"><span class="keyword">fn</span> <span class="title function_">main</span>() {</span><br><span class="line"> <span class="keyword">let</span> <span class="variable">reference_to_nothing</span> = <span class="title function_ invoke__">dangle</span>();</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">fn</span> <span class="title function_">dangle</span>() <span class="punctuation">-></span> &<span class="type">String</span> {</span><br><span class="line"> <span class="keyword">let</span> <span class="variable">s</span> = <span class="type">String</span>::<span class="title function_ invoke__">from</span>(<span class="string">"hello"</span>);</span><br><span class="line"> &s</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里可以看到其实在 dangle函数返回后，这里的 s 理论上就离开了作用域，但是由于返回了 s 的引用，在 main 函数中就会拿着这个引用，就会出现如下错误<br><img data-src="https://img.nicksxs.me/uPic/JtrXSW.png"></p>
|
|
<h3 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h3><p>最后总结下</p>
|
|
<ul>
|
|
<li>在任何一个段给定的时间里，你要么只能拥有一个可变引用，要么只能拥有任意数量的不可变引用。</li>
|
|
<li>引用总是有效的。</li>
|
|
</ul>
|
|
]]></content>
|
|
<categories>
|
|
<category>语言</category>
|
|
<category>Rust</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Rust</tag>
|
|
<tag>所有权</tag>
|
|
<tag>内存分布</tag>
|
|
<tag>新语言</tag>
|
|
<tag>可变引用</tag>
|
|
<tag>不可变引用</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>spark-little-tips</title>
|
|
<url>/2017/03/28/spark-little-tips/</url>
|
|
<content><![CDATA[<h2 id="spark-的一些粗浅使用经验"><a href="#spark-的一些粗浅使用经验" class="headerlink" title="spark 的一些粗浅使用经验"></a>spark 的一些粗浅使用经验</h2><p>工作中学习使用了一下Spark做数据分析,主要是用spark的python接口，首先是<code>pyspark.SparkContext(appName=xxx)</code>，这是初始化一个Spark应用实例或者说会话，不能重复，<br>返回的实例句柄就可以调用<code>textFile(path)</code>读取文本文件，这里的文本文件可以是HDFS上的文本文件，也可以普通文本文件，但是需要在Spark的所有集群上都存在，否则会<br>读取失败，<code>parallelize</code>则可以将python生成的集合数据读取后转换成rdd(A Resilient Distributed Dataset (RDD),一种spark下的基本抽象数据集),基于这个RDD就可以做<br>数据的流式计算，例如<code>map reduce</code>,在Spark中可以非常方便地实现 </p>
|
|
<h3 id="简单的mapreduce-word-count示例"><a href="#简单的mapreduce-word-count示例" class="headerlink" title="简单的mapreduce word count示例"></a>简单的mapreduce word count示例</h3><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">textFile = sc.parallelize([(<span class="number">1</span>,<span class="number">1</span>), (<span class="number">2</span>,<span class="number">1</span>), (<span class="number">3</span>,<span class="number">1</span>), (<span class="number">4</span>,<span class="number">1</span>), (<span class="number">5</span>,<span class="number">1</span>),(<span class="number">1</span>,<span class="number">1</span>), (<span class="number">2</span>,<span class="number">1</span>), (<span class="number">3</span>,<span class="number">1</span>), (<span class="number">4</span>,<span class="number">1</span>), (<span class="number">5</span>,<span class="number">1</span>)])</span><br><span class="line">data = textFile.reduceByKey(<span class="keyword">lambda</span> x, y: x + y).collect()</span><br><span class="line"><span class="keyword">for</span> _ <span class="keyword">in</span> data:</span><br><span class="line"> <span class="built_in">print</span>(_)</span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
<h3 id="结果"><a href="#结果" class="headerlink" title="结果"></a>结果</h3><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">(3, 2)</span><br><span class="line">(1, 2)</span><br><span class="line">(4, 2)</span><br><span class="line">(2, 2)</span><br><span class="line">(5, 2)</span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>data analysis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>spark</tag>
|
|
<tag>python</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>spring boot中的 http 接口返回 json 形式的小注意点</title>
|
|
<url>/2023/06/25/spring-boot%E4%B8%AD%E7%9A%84-http-%E6%8E%A5%E5%8F%A3%E8%BF%94%E5%9B%9E-json-%E5%BD%A2%E5%BC%8F%E7%9A%84%E5%B0%8F%E6%B3%A8%E6%84%8F%E7%82%B9/</url>
|
|
<content><![CDATA[<p>这个可能是个很简单的点，不过之前碰到了就记录下，我们常规的应用都是使用统一的请求响应转换器去处理请求和响应返回，但是对于有文件上传或者返回的是文件的情况，一般都是不使用统一的处理，但是在响应返回的时候可能会存在这样的情况，如果文件正常被处理那就返回文件，如果处理异常需要给前端返回 json类型的响应，里面能够取到响应码错误描述等</p>
|
|
<p>比如在请求中参数就使用 <code>httpRequest(HttpServletRequest request, HttpServletResponse response)</code><br>然后在返回的时候就使用 <code>response.getOutputStream().write(result)</code>，而如果是要返回 json 形式的话就可以像这个文章说明的<br><a href="https://www.baeldung.com/servlet-json-response">链接</a></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">Employee</span> <span class="variable">employee</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Employee</span>(<span class="number">1</span>, <span class="string">"Karan"</span>, <span class="string">"IT"</span>, <span class="number">5000</span>);</span><br><span class="line"><span class="type">String</span> <span class="variable">employeeJsonString</span> <span class="operator">=</span> <span class="built_in">this</span>.gson.toJson(employee);</span><br><span class="line"></span><br><span class="line"><span class="type">PrintWriter</span> <span class="variable">out</span> <span class="operator">=</span> response.getWriter();</span><br><span class="line">response.setContentType(<span class="string">"application/json"</span>);</span><br><span class="line">response.setCharacterEncoding(<span class="string">"UTF-8"</span>);</span><br><span class="line">out.print(employeeJsonString);</span><br><span class="line">out.flush(); </span><br></pre></td></tr></table></figure>
|
|
<p>一开始我也是这么一搜就用了，后来发现返回的一直是乱码，仔细看了下发现了个问题，就是这个 response 设置 contentType 是在<code>getWriter</code>之后的，这样自然就不会起作用了，所以要在设置 <code>setContentType</code> 和 <code>setCharacterEncoding</code> 之后再 <code>getWriter</code>，之后就可以正常返回了。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Spring</tag>
|
|
<tag>SpringBoot</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>spring event 介绍</title>
|
|
<url>/2022/01/30/spring-event-%E4%BB%8B%E7%BB%8D/</url>
|
|
<content><![CDATA[<p>spring框架中如果想使用一些一部操作，除了依赖第三方中间件的消息队列，还可以用spring自己的event，简单介绍下使用方法<br>首先我们可以建一个event，继承ApplicationEvent</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">CustomSpringEvent</span> <span class="keyword">extends</span> <span class="title class_">ApplicationEvent</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> String message;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">CustomSpringEvent</span><span class="params">(Object source, String message)</span> {</span><br><span class="line"> <span class="built_in">super</span>(source);</span><br><span class="line"> <span class="built_in">this</span>.message = message;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">public</span> String <span class="title function_">getMessage</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> message;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这个 ApplicationEvent 其实也比较简单，内部就一个 Object 类型的 source，可以自行扩展，我们在自定义的这个 Event 里加了个 Message ，只是简单介绍下使用</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">abstract</span> <span class="keyword">class</span> <span class="title class_">ApplicationEvent</span> <span class="keyword">extends</span> <span class="title class_">EventObject</span> {</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">long</span> <span class="variable">serialVersionUID</span> <span class="operator">=</span> <span class="number">7099057708183571937L</span>;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="type">long</span> timestamp;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">ApplicationEvent</span><span class="params">(Object source)</span> {</span><br><span class="line"> <span class="built_in">super</span>(source);</span><br><span class="line"> <span class="built_in">this</span>.timestamp = System.currentTimeMillis();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">ApplicationEvent</span><span class="params">(Object source, Clock clock)</span> {</span><br><span class="line"> <span class="built_in">super</span>(source);</span><br><span class="line"> <span class="built_in">this</span>.timestamp = clock.millis();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">final</span> <span class="type">long</span> <span class="title function_">getTimestamp</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.timestamp;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后就是事件生产者和监听消费者</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Component</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">CustomSpringEventPublisher</span> {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Resource</span></span><br><span class="line"> <span class="keyword">private</span> ApplicationEventPublisher applicationEventPublisher;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">publishCustomEvent</span><span class="params">(<span class="keyword">final</span> String message)</span> {</span><br><span class="line"> System.out.println(<span class="string">"Publishing custom event. "</span>);</span><br><span class="line"> <span class="type">CustomSpringEvent</span> <span class="variable">customSpringEvent</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">CustomSpringEvent</span>(<span class="built_in">this</span>, message);</span><br><span class="line"> applicationEventPublisher.publishEvent(customSpringEvent);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里的 ApplicationEventPublisher 是 Spring 的方法接口</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@FunctionalInterface</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">interface</span> <span class="title class_">ApplicationEventPublisher</span> {</span><br><span class="line"> <span class="keyword">default</span> <span class="keyword">void</span> <span class="title function_">publishEvent</span><span class="params">(ApplicationEvent event)</span> {</span><br><span class="line"> <span class="built_in">this</span>.publishEvent((Object)event);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">void</span> <span class="title function_">publishEvent</span><span class="params">(Object var1)</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>具体的是例如 <code>org.springframework.context.support.AbstractApplicationContext#publishEvent(java.lang.Object, org.springframework.core.ResolvableType)</code> 中的实现，后面可以展开讲讲</p>
|
|
<p>事件监听者：</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Component</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">CustomSpringEventListener</span> <span class="keyword">implements</span> <span class="title class_">ApplicationListener</span><CustomSpringEvent> {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">onApplicationEvent</span><span class="params">(CustomSpringEvent event)</span> {</span><br><span class="line"> System.out.println(<span class="string">"Received spring custom event - "</span> + event.getMessage());</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里的也是 spring 的一个方法接口</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@FunctionalInterface</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">interface</span> <span class="title class_">ApplicationListener</span><E <span class="keyword">extends</span> <span class="title class_">ApplicationEvent</span>> <span class="keyword">extends</span> <span class="title class_">EventListener</span> {</span><br><span class="line"> <span class="keyword">void</span> <span class="title function_">onApplicationEvent</span><span class="params">(E var1)</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">static</span> <T> ApplicationListener<PayloadApplicationEvent<T>> <span class="title function_">forPayload</span><span class="params">(Consumer<T> consumer)</span> {</span><br><span class="line"> <span class="keyword">return</span> (event) -> {</span><br><span class="line"> consumer.accept(event.getPayload());</span><br><span class="line"> };</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后简单包个请求</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"></span><br><span class="line"><span class="meta">@RequestMapping(value = "/event", method = RequestMethod.GET)</span></span><br><span class="line"><span class="meta">@ResponseBody</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">event</span><span class="params">()</span> {</span><br><span class="line"> customSpringEventPublisher.publishCustomEvent(<span class="string">"hello sprint event"</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p><img data-src="https://img.nicksxs.me/uPic/RppSmk.png"><br>就能看到接收到消息了。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Spring</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Spring</tag>
|
|
<tag>Spring Event</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>springboot mappings 注册逻辑</title>
|
|
<url>/2023/08/13/springboot-mappings-%E6%B3%A8%E5%86%8C%E9%80%BB%E8%BE%91/</url>
|
|
<content><![CDATA[<p>前面讲了怎么获取 mapping url，继续说下这些mappings 是怎么注册进去的，<br>来看下这个 <code>RequestMappingHandlerMapping</code> 的继承关系<br><img data-src="https://img.nicksxs.me/blog/76d96T.png"><br>可以看到这个类实现了 <code>org.springframework.beans.factory.InitializingBean</code> 这个接口，然后这个 <code>InitializingBean</code> 提供了 <code>org.springframework.beans.factory.InitializingBean#afterPropertiesSet</code> 接口，可以在 bean 初始化后做一些属性设置等，<br>这里是调用了类本身的 afterPropertiesSet 方法和父类的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">afterPropertiesSet</span><span class="params">()</span> {</span><br><span class="line"> <span class="built_in">this</span>.config = <span class="keyword">new</span> <span class="title class_">RequestMappingInfo</span>.BuilderConfiguration();</span><br><span class="line"> <span class="built_in">this</span>.config.setUrlPathHelper(getUrlPathHelper());</span><br><span class="line"> <span class="built_in">this</span>.config.setPathMatcher(getPathMatcher());</span><br><span class="line"> <span class="built_in">this</span>.config.setSuffixPatternMatch(useSuffixPatternMatch());</span><br><span class="line"> <span class="built_in">this</span>.config.setTrailingSlashMatch(useTrailingSlashMatch());</span><br><span class="line"> <span class="built_in">this</span>.config.setRegisteredSuffixPatternMatch(useRegisteredSuffixPatternMatch());</span><br><span class="line"> <span class="built_in">this</span>.config.setContentNegotiationManager(getContentNegotiationManager());</span><br><span class="line"></span><br><span class="line"> <span class="built_in">super</span>.afterPropertiesSet();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>父类是 <code>org.springframework.web.servlet.handler.AbstractHandlerMethodMapping#afterPropertiesSet</code><br>具体代码很简略，就是初始化 HandlerMethod</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">afterPropertiesSet</span><span class="params">()</span> {</span><br><span class="line"> initHandlerMethods();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>也就是调用了 <code>org.springframework.web.servlet.handler.AbstractHandlerMethodMapping#initHandlerMethods</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">initHandlerMethods</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">for</span> (String beanName : getCandidateBeanNames()) {</span><br><span class="line"> <span class="keyword">if</span> (!beanName.startsWith(SCOPED_TARGET_NAME_PREFIX)) {</span><br><span class="line"> processCandidateBean(beanName);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> handlerMethodsInitialized(getHandlerMethods());</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后就是调用 <code>org.springframework.web.servlet.handler.AbstractHandlerMethodMapping#processCandidateBean</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">processCandidateBean</span><span class="params">(String beanName)</span> {</span><br><span class="line"> Class<?> beanType = <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> beanType = obtainApplicationContext().getType(beanName);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">catch</span> (Throwable ex) {</span><br><span class="line"> <span class="comment">// An unresolvable bean type, probably from a lazy bean - let's ignore it.</span></span><br><span class="line"> <span class="keyword">if</span> (logger.isTraceEnabled()) {</span><br><span class="line"> logger.trace(<span class="string">"Could not resolve type for bean '"</span> + beanName + <span class="string">"'"</span>, ex);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (beanType != <span class="literal">null</span> && isHandler(beanType)) {</span><br><span class="line"> detectHandlerMethods(beanName);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>先是获取的 beanType，在判断 beanType 是不是 Handler，通过方法 <code>org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping#isHandler</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">protected</span> <span class="type">boolean</span> <span class="title function_">isHandler</span><span class="params">(Class<?> beanType)</span> {</span><br><span class="line"> <span class="keyword">return</span> (AnnotatedElementUtils.hasAnnotation(beanType, Controller.class) ||</span><br><span class="line"> AnnotatedElementUtils.hasAnnotation(beanType, RequestMapping.class));</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>就很简单，判断是不是有 Controller 注解或者 RequestMapping 注解<br>然后就是判断 HandlerMethod 了，调用了 <code>org.springframework.web.servlet.handler.AbstractHandlerMethodMapping#detectHandlerMethods</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">detectHandlerMethods</span><span class="params">(Object handler)</span> {</span><br><span class="line"> Class<?> handlerType = (handler <span class="keyword">instanceof</span> String ?</span><br><span class="line"> obtainApplicationContext().getType((String) handler) : handler.getClass());</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (handlerType != <span class="literal">null</span>) {</span><br><span class="line"> Class<?> userType = ClassUtils.getUserClass(handlerType);</span><br><span class="line"> Map<Method, T> methods = MethodIntrospector.selectMethods(userType,</span><br><span class="line"> (MethodIntrospector.MetadataLookup<T>) method -> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">return</span> getMappingForMethod(method, userType);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">catch</span> (Throwable ex) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalStateException</span>(<span class="string">"Invalid mapping on handler class ["</span> +</span><br><span class="line"> userType.getName() + <span class="string">"]: "</span> + method, ex);</span><br><span class="line"> }</span><br><span class="line"> });</span><br><span class="line"> <span class="keyword">if</span> (logger.isTraceEnabled()) {</span><br><span class="line"> logger.trace(formatMappings(userType, methods));</span><br><span class="line"> }</span><br><span class="line"> methods.forEach((method, mapping) -> {</span><br><span class="line"> <span class="type">Method</span> <span class="variable">invocableMethod</span> <span class="operator">=</span> AopUtils.selectInvocableMethod(method, userType);</span><br><span class="line"> registerHandlerMethod(handler, invocableMethod, mapping);</span><br><span class="line"> });</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>前面先通过 getMappingForMethod 找出有 Mapping 的方法，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="meta">@Nullable</span></span><br><span class="line"><span class="keyword">protected</span> RequestMappingInfo <span class="title function_">getMappingForMethod</span><span class="params">(Method method, Class<?> handlerType)</span> {</span><br><span class="line"> <span class="type">RequestMappingInfo</span> <span class="variable">info</span> <span class="operator">=</span> createRequestMappingInfo(method);</span><br><span class="line"> <span class="keyword">if</span> (info != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">RequestMappingInfo</span> <span class="variable">typeInfo</span> <span class="operator">=</span> createRequestMappingInfo(handlerType);</span><br><span class="line"> <span class="keyword">if</span> (typeInfo != <span class="literal">null</span>) {</span><br><span class="line"> info = typeInfo.combine(info);</span><br><span class="line"> }</span><br><span class="line"> <span class="type">String</span> <span class="variable">prefix</span> <span class="operator">=</span> getPathPrefix(handlerType);</span><br><span class="line"> <span class="keyword">if</span> (prefix != <span class="literal">null</span>) {</span><br><span class="line"> info = RequestMappingInfo.paths(prefix).options(<span class="built_in">this</span>.config).build().combine(info);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> info;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Nullable</span></span><br><span class="line"><span class="keyword">private</span> RequestMappingInfo <span class="title function_">createRequestMappingInfo</span><span class="params">(AnnotatedElement element)</span> {</span><br><span class="line"> <span class="type">RequestMapping</span> <span class="variable">requestMapping</span> <span class="operator">=</span> AnnotatedElementUtils.findMergedAnnotation(element, RequestMapping.class);</span><br><span class="line"> RequestCondition<?> condition = (element <span class="keyword">instanceof</span> Class ?</span><br><span class="line"> getCustomTypeCondition((Class<?>) element) : getCustomMethodCondition((Method) element));</span><br><span class="line"> <span class="keyword">return</span> (requestMapping != <span class="literal">null</span> ? createRequestMappingInfo(requestMapping, condition) : <span class="literal">null</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后再是对 Method 循环调用 <code>org.springframework.web.servlet.handler.AbstractHandlerMethodMapping#registerHandlerMethod</code><br>可以看到就是上一篇的 <code>org.springframework.web.servlet.handler.AbstractHandlerMethodMapping#mappingRegistry</code> 去存储映射信息</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">registerHandlerMethod</span><span class="params">(Object handler, Method method, T mapping)</span> {</span><br><span class="line"> <span class="built_in">this</span>.mappingRegistry.register(mapping, handler, method);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>最后是真的注册逻辑</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">register</span><span class="params">(T mapping, Object handler, Method method)</span> {</span><br><span class="line"> <span class="comment">// Assert that the handler method is not a suspending one.</span></span><br><span class="line"> <span class="keyword">if</span> (KotlinDetector.isKotlinType(method.getDeclaringClass())) {</span><br><span class="line"> Class<?>[] parameterTypes = method.getParameterTypes();</span><br><span class="line"> <span class="keyword">if</span> ((parameterTypes.length > <span class="number">0</span>) && <span class="string">"kotlin.coroutines.Continuation"</span>.equals(parameterTypes[parameterTypes.length - <span class="number">1</span>].getName())) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalStateException</span>(<span class="string">"Unsupported suspending handler method detected: "</span> + method);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">this</span>.readWriteLock.writeLock().lock();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">HandlerMethod</span> <span class="variable">handlerMethod</span> <span class="operator">=</span> createHandlerMethod(handler, method);</span><br><span class="line"> validateMethodMapping(handlerMethod, mapping);</span><br><span class="line"> <span class="built_in">this</span>.mappingLookup.put(mapping, handlerMethod);</span><br><span class="line"></span><br><span class="line"> List<String> directUrls = getDirectUrls(mapping);</span><br><span class="line"> <span class="keyword">for</span> (String url : directUrls) {</span><br><span class="line"> <span class="built_in">this</span>.urlLookup.add(url, mapping);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">String</span> <span class="variable">name</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">if</span> (getNamingStrategy() != <span class="literal">null</span>) {</span><br><span class="line"> name = getNamingStrategy().getName(handlerMethod, mapping);</span><br><span class="line"> addMappingName(name, handlerMethod);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">CorsConfiguration</span> <span class="variable">corsConfig</span> <span class="operator">=</span> initCorsConfiguration(handler, method, mapping);</span><br><span class="line"> <span class="keyword">if</span> (corsConfig != <span class="literal">null</span>) {</span><br><span class="line"> <span class="built_in">this</span>.corsLookup.put(handlerMethod, corsConfig);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.registry.put(mapping, <span class="keyword">new</span> <span class="title class_">MappingRegistration</span><>(mapping, handlerMethod, directUrls, name));</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">finally</span> {</span><br><span class="line"> <span class="built_in">this</span>.readWriteLock.writeLock().unlock();</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>底层的存储就是上一篇说的 mappingLookup 来存储信息</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>SpringBoot</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>java的agent继续体验</title>
|
|
<url>/2024/12/29/java%E7%9A%84agent%E7%BB%A7%E7%BB%AD%E4%BD%93%E9%AA%8C/</url>
|
|
<content><![CDATA[<p>在上次的基础上我们可以通过一些方法来获取参数的参数名，以此我们又可以达到类似于切面的功能逻辑，<br>首先我在之前的代码里做一点修改</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">start</span><span class="params">(String input)</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> System.out.println(<span class="string">"Demo starting ..."</span>);</span><br><span class="line"> System.out.println(input);</span><br><span class="line"> Thread.sleep(<span class="number">123</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException ignore) {</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>就是带了个参数，这边是我把参数打了出来，如果没有打，我想通过agent来知道这个参数值是啥就可以用agent技术<br>获取代码中的变量名以及类型</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">MethodInfo</span> <span class="variable">methodInfo</span> <span class="operator">=</span> declaredMethod.getMethodInfo();</span><br><span class="line"><span class="type">CodeAttribute</span> <span class="variable">codeAttribute</span> <span class="operator">=</span> methodInfo.getCodeAttribute();</span><br><span class="line"><span class="type">LocalVariableAttribute</span> <span class="variable">attribute</span> <span class="operator">=</span> (LocalVariableAttribute) codeAttribute.getAttribute(LocalVariableAttribute.tag);</span><br><span class="line">System.out.println(attribute.variableName(<span class="number">0</span>));</span><br></pre></td></tr></table></figure>
|
|
<p>这边就是个string类型，我们就能够获取到 <code>input</code> 这个变量名，然后我们就可以在这里插入一段代码来打印这个参数</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">declaredMethod.insertAfter(<span class="string">"System.out.println( "</span> + attribute.variableName(<span class="number">0</span>)+ <span class="string">" );"</span>);</span><br></pre></td></tr></table></figure>
|
|
<p>这样我们就能做到类似于java中切面的技术</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>springboot web server 启动逻辑</title>
|
|
<url>/2023/08/20/springboot-web-server-%E5%90%AF%E5%8A%A8%E9%80%BB%E8%BE%91/</url>
|
|
<content><![CDATA[<p>springboot 的一个方便之处就是集成了 web server 进去，接着上一篇继续来看下这个 web server 的启动过程<br>这边是基于 springboot 的 2.2.9.RELEASE 版本，整个 springboot 体系主体就是看<br><code>org.springframework.context.support.AbstractApplicationContext#refresh</code><br>刷新方法，而启动 web server 的方法就是在其中的 onRefresh </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// Allows post-processing of the bean factory in context subclasses.</span></span><br><span class="line"> postProcessBeanFactory(beanFactory);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Invoke factory processors registered as beans in the context.</span></span><br><span class="line"> invokeBeanFactoryPostProcessors(beanFactory);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Register bean processors that intercept bean creation.</span></span><br><span class="line"> registerBeanPostProcessors(beanFactory);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Initialize message source for this context.</span></span><br><span class="line"> initMessageSource();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Initialize event multicaster for this context.</span></span><br><span class="line"> initApplicationEventMulticaster();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Initialize other special beans in specific context subclasses.</span></span><br><span class="line"> <span class="comment">// ------------------> ‼️就是这里</span></span><br><span class="line"> onRefresh();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Check for listener beans and register them.</span></span><br><span class="line"> registerListeners();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Instantiate all remaining (non-lazy-init) singletons.</span></span><br><span class="line"> finishBeanFactoryInitialization(beanFactory);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Last step: publish corresponding event.</span></span><br><span class="line"> finishRefresh();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>跟进去其实本身是啥都不做的，由具体的实现类子类去做各自的处理<br><code>org.springframework.context.support.AbstractApplicationContext#onRefresh</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Template method which can be overridden to add context-specific refresh work.</span></span><br><span class="line"><span class="comment"> * Called on initialization of special beans, before instantiation of singletons.</span></span><br><span class="line"><span class="comment"> * <p>This implementation is empty.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@throws</span> BeansException in case of errors</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> #refresh()</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">onRefresh</span><span class="params">()</span> <span class="keyword">throws</span> BeansException {</span><br><span class="line"> <span class="comment">// For subclasses: do nothing by default.</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>看具体的实现类里就有我们的主角<br><code>org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext#onRefresh</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">onRefresh</span><span class="params">()</span> {</span><br><span class="line"> <span class="built_in">super</span>.onRefresh();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// -------------> 主要就是这里，创建 webserver</span></span><br><span class="line"> <span class="built_in">this</span>.createWebServer();</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable var2) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">ApplicationContextException</span>(<span class="string">"Unable to start web server"</span>, var2);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>具体的就是一些前置处理<br><code>org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext#createWebServer</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">createWebServer</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">WebServer</span> <span class="variable">webServer</span> <span class="operator">=</span> <span class="built_in">this</span>.webServer;</span><br><span class="line"> <span class="type">ServletContext</span> <span class="variable">servletContext</span> <span class="operator">=</span> <span class="built_in">this</span>.getServletContext();</span><br><span class="line"> <span class="keyword">if</span> (webServer == <span class="literal">null</span> && servletContext == <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">ServletWebServerFactory</span> <span class="variable">factory</span> <span class="operator">=</span> <span class="built_in">this</span>.getWebServerFactory();</span><br><span class="line"> <span class="built_in">this</span>.webServer = factory.getWebServer(<span class="keyword">new</span> <span class="title class_">ServletContextInitializer</span>[]{<span class="built_in">this</span>.getSelfInitializer()});</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (servletContext != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="built_in">this</span>.getSelfInitializer().onStartup(servletContext);</span><br><span class="line"> } <span class="keyword">catch</span> (ServletException var4) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">ApplicationContextException</span>(<span class="string">"Cannot initialize servlet context"</span>, var4);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.initPropertySources();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>初始的 webServer 和 servletContext 都是 null，需要进行初始化<br>先是获取 WebServer 工厂,<br><code>org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext#getWebServerFactory</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> ServletWebServerFactory <span class="title function_">getWebServerFactory</span><span class="params">()</span> {</span><br><span class="line"> String[] beanNames = <span class="built_in">this</span>.getBeanFactory().getBeanNamesForType(ServletWebServerFactory.class);</span><br><span class="line"> <span class="keyword">if</span> (beanNames.length == <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">ApplicationContextException</span>(<span class="string">"Unable to start ServletWebServerApplicationContext due to missing ServletWebServerFactory bean."</span>);</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (beanNames.length > <span class="number">1</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">ApplicationContextException</span>(<span class="string">"Unable to start ServletWebServerApplicationContext due to multiple ServletWebServerFactory beans : "</span> + StringUtils.arrayToCommaDelimitedString(beanNames));</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> (ServletWebServerFactory)<span class="built_in">this</span>.getBeanFactory().getBean(beanNames[<span class="number">0</span>], ServletWebServerFactory.class);</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
<p>获取了类型是 <code>ServletWebServerFactory</code> 的 bean，然后再回来就是调用的接口方法<br><code>org.springframework.boot.web.servlet.server.ServletWebServerFactory#getWebServer</code><br>根据具体的 factory 来生成对应的譬如 tomcat 的 factory，<br><code>org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory#getWebServer</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> WebServer <span class="title function_">getWebServer</span><span class="params">(ServletContextInitializer... initializers)</span> {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.disableMBeanRegistry) {</span><br><span class="line"> Registry.disableRegistry();</span><br><span class="line"> }</span><br><span class="line"> <span class="type">Tomcat</span> <span class="variable">tomcat</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Tomcat</span>();</span><br><span class="line"> <span class="type">File</span> <span class="variable">baseDir</span> <span class="operator">=</span> (<span class="built_in">this</span>.baseDirectory != <span class="literal">null</span>) ? <span class="built_in">this</span>.baseDirectory : createTempDir(<span class="string">"tomcat"</span>);</span><br><span class="line"> tomcat.setBaseDir(baseDir.getAbsolutePath());</span><br><span class="line"> <span class="type">Connector</span> <span class="variable">connector</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Connector</span>(<span class="built_in">this</span>.protocol);</span><br><span class="line"> connector.setThrowOnFailure(<span class="literal">true</span>);</span><br><span class="line"> tomcat.getService().addConnector(connector);</span><br><span class="line"> customizeConnector(connector);</span><br><span class="line"> tomcat.setConnector(connector);</span><br><span class="line"> tomcat.getHost().setAutoDeploy(<span class="literal">false</span>);</span><br><span class="line"> configureEngine(tomcat.getEngine());</span><br><span class="line"> <span class="keyword">for</span> (Connector additionalConnector : <span class="built_in">this</span>.additionalTomcatConnectors) {</span><br><span class="line"> tomcat.getService().addConnector(additionalConnector);</span><br><span class="line"> }</span><br><span class="line"> prepareContext(tomcat.getHost(), initializers);</span><br><span class="line"> <span class="keyword">return</span> getTomcatWebServer(tomcat);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/blog/ApIStM.jpg"></p>
|
|
<p>最后一行就是创建 TomcatWebServer,<br><code>org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory#getTomcatWebServer</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Factory method called to create the {<span class="doctag">@link</span> TomcatWebServer}. Subclasses can</span></span><br><span class="line"><span class="comment"> * override this method to return a different {<span class="doctag">@link</span> TomcatWebServer} or apply</span></span><br><span class="line"><span class="comment"> * additional processing to the Tomcat server.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> tomcat the Tomcat server.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> a new {<span class="doctag">@link</span> TomcatWebServer} instance</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">protected</span> TomcatWebServer <span class="title function_">getTomcatWebServer</span><span class="params">(Tomcat tomcat)</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">TomcatWebServer</span>(tomcat, getPort() >= <span class="number">0</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里面就开始 new 了一个 TomcatWebServer，<br><code>org.springframework.boot.web.embedded.tomcat.TomcatWebServer#TomcatWebServer(org.apache.catalina.startup.Tomcat, boolean)</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">TomcatWebServer</span><span class="params">(Tomcat tomcat, <span class="type">boolean</span> autoStart)</span> {</span><br><span class="line"> Assert.notNull(tomcat, <span class="string">"Tomcat Server must not be null"</span>);</span><br><span class="line"> <span class="built_in">this</span>.tomcat = tomcat;</span><br><span class="line"> <span class="built_in">this</span>.autoStart = autoStart;</span><br><span class="line"> initialize();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>再调用里面的初始化方法，<br><code>org.springframework.boot.web.embedded.tomcat.TomcatWebServer#initialize</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">initialize</span><span class="params">()</span> <span class="keyword">throws</span> WebServerException {</span><br><span class="line"> logger.info(<span class="string">"Tomcat initialized with port(s): "</span> + getPortsDescription(<span class="literal">false</span>));</span><br><span class="line"> <span class="keyword">synchronized</span> (<span class="built_in">this</span>.monitor) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> addInstanceIdToEngineName();</span><br><span class="line"></span><br><span class="line"> <span class="type">Context</span> <span class="variable">context</span> <span class="operator">=</span> findContext();</span><br><span class="line"> context.addLifecycleListener((event) -> {</span><br><span class="line"> <span class="keyword">if</span> (context.equals(event.getSource()) && Lifecycle.START_EVENT.equals(event.getType())) {</span><br><span class="line"> <span class="comment">// Remove service connectors so that protocol binding doesn't</span></span><br><span class="line"> <span class="comment">// happen when the service is started.</span></span><br><span class="line"> removeServiceConnectors();</span><br><span class="line"> }</span><br><span class="line"> });</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Start the server to trigger initialization listeners</span></span><br><span class="line"> <span class="built_in">this</span>.tomcat.start();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// We can re-throw failure exception directly in the main thread</span></span><br><span class="line"> rethrowDeferredStartupExceptions();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> ContextBindings.bindClassLoader(context, context.getNamingToken(), getClass().getClassLoader());</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">catch</span> (NamingException ex) {</span><br><span class="line"> <span class="comment">// Naming is not enabled. Continue</span></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Unlike Jetty, all Tomcat threads are daemon threads. We create a</span></span><br><span class="line"> <span class="comment">// blocking non-daemon to stop immediate shutdown</span></span><br><span class="line"> startDaemonAwaitThread();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">catch</span> (Exception ex) {</span><br><span class="line"> stopSilently();</span><br><span class="line"> destroySilently();</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">WebServerException</span>(<span class="string">"Unable to start embedded Tomcat"</span>, ex);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">``` </span><br><span class="line">这里就是继续调用 `org.apache.catalina.startup.Tomcat#start`</span><br><span class="line">```java</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">start</span><span class="params">()</span> <span class="keyword">throws</span> LifecycleException {</span><br><span class="line"> <span class="built_in">this</span>.getServer();</span><br><span class="line"> <span class="built_in">this</span>.server.start();</span><br><span class="line"> }</span><br><span class="line">``` </span><br><span class="line">获取server, `org.apache.catalina.startup.Tomcat#getServer`</span><br><span class="line">```java</span><br><span class="line"><span class="keyword">public</span> Server <span class="title function_">getServer</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.server != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.server;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> System.setProperty(<span class="string">"catalina.useNaming"</span>, <span class="string">"false"</span>);</span><br><span class="line"> <span class="built_in">this</span>.server = <span class="keyword">new</span> <span class="title class_">StandardServer</span>();</span><br><span class="line"> <span class="built_in">this</span>.initBaseDir();</span><br><span class="line"> ConfigFileLoader.setSource(<span class="keyword">new</span> <span class="title class_">CatalinaBaseConfigurationSource</span>(<span class="keyword">new</span> <span class="title class_">File</span>(<span class="built_in">this</span>.basedir), (String)<span class="literal">null</span>));</span><br><span class="line"> <span class="built_in">this</span>.server.setPort(-<span class="number">1</span>);</span><br><span class="line"> <span class="type">Service</span> <span class="variable">service</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">StandardService</span>();</span><br><span class="line"> service.setName(<span class="string">"Tomcat"</span>);</span><br><span class="line"> <span class="built_in">this</span>.server.addService(service);</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.server;</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后就是启动 server，后面可以继续看这个启动 TomcatServer 内部的逻辑</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>SpringBoot</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>springboot 处理请求的小分支-跳转 & cookie</title>
|
|
<url>/2023/09/03/springboot-%E5%A4%84%E7%90%86%E8%AF%B7%E6%B1%82%E7%9A%84%E5%B0%8F%E5%88%86%E6%94%AF-%E8%B7%B3%E8%BD%AC-cookie/</url>
|
|
<content><![CDATA[<p>首先是跳转，应该说设置状态，其中跳转是其中一个状态，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@RequestMapping(value = "request1", method = RequestMethod.GET)</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">request1</span><span class="params">(HttpServletRequest request, HttpServletResponse response)</span> {</span><br><span class="line"> response.setStatus(HttpServletResponse.SC_OK);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>200 状态就很简单，如果想做跳转可以使用 302 并设置 location</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@RequestMapping(value = "request2", method = RequestMethod.GET)</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">request2</span><span class="params">(HttpServletRequest request, HttpServletResponse response)</span> {</span><br><span class="line"> response.setHeader(<span class="string">"Location"</span>, <span class="string">"https://baidu.com"</span>);</span><br><span class="line"> response.setStatus(HttpServletResponse.SC_MOVED_TEMPORARILY);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里如果设置了 302，但没设置跳转 location，就是 302 found</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">int</span> <span class="variable">SC_MOVED_TEMPORARILY</span> <span class="operator">=</span> <span class="number">302</span>;</span><br><span class="line"><span class="type">int</span> <span class="variable">SC_FOUND</span> <span class="operator">=</span> <span class="number">302</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>而如果是 301 跳转，代表原地址不可用了，永久跳转</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@RequestMapping(value = "request3", method = RequestMethod.GET)</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">request3</span><span class="params">(HttpServletRequest request, HttpServletResponse response)</span> {</span><br><span class="line"> response.setHeader(<span class="string">"Location"</span>, <span class="string">"https://baidu.com"</span>);</span><br><span class="line"> response.setStatus(HttpServletResponse.SC_MOVED_PERMANENTLY);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<h1 id="另一个问题是设置-cookie"><a href="#另一个问题是设置-cookie" class="headerlink" title="另一个问题是设置 cookie"></a>另一个问题是设置 cookie</h1><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@RequestMapping(value = "request4", method = RequestMethod.GET)</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">request4</span><span class="params">(HttpServletRequest request, HttpServletResponse response)</span> {</span><br><span class="line"> <span class="type">Cookie</span> <span class="variable">cookie</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Cookie</span>(<span class="string">"a"</span>, <span class="string">"b"</span>);</span><br><span class="line"> response.addCookie(cookie);</span><br><span class="line"> response.setStatus(HttpServletResponse.SC_OK);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这样的设置就能成功设定 cookie，而随着目前的浏览器 cookie 策略，如果要跳转后设置的话，估计是会越来越难，包括</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">response.addHeader(<span class="string">"Set-Cookie"</span>, <span class="string">"a=b; domain=baidu.com; SameSite=none;Secure"</span>);</span><br></pre></td></tr></table></figure>
|
|
<p>这样的方式也没法实现。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>SpringBoot</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>springboot 获取 web 应用中所有的接口 url</title>
|
|
<url>/2023/08/06/springboot-%E8%8E%B7%E5%8F%96-web-%E5%BA%94%E7%94%A8%E4%B8%AD%E6%89%80%E6%9C%89%E7%9A%84%E6%8E%A5%E5%8F%A3-url/</url>
|
|
<content><![CDATA[<p>最近有个小需求，要把我们一个 springboot 应用的 request mapping 给导出来，这么说已经是转化过了的，应该是要整理这个应用所有的接口路径，比如我有一个 api.baidu1.com 作为接口域名，然后这个域名下有很多个接口提供服务，这些接口都是写在一个 springboot 应用里，如果本身有网关管理这些接口转发的其实可以通过网关的数据输出，这里就介绍下通过代码来获取</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">RequestMappingHandlerMapping</span> <span class="variable">mapping</span> <span class="operator">=</span> appContext.getBean(RequestMappingHandlerMapping.class);</span><br><span class="line">Map<RequestMappingInfo, HandlerMethod> map = mapping.getHandlerMethods();</span><br><span class="line"><span class="keyword">for</span> (Map.Entry<RequestMappingInfo, HandlerMethod> entry : map.entrySet()) {</span><br><span class="line"> Set<String> urlSet = entry.getKey().getPatternsCondition().getPatterns();</span><br><span class="line"> <span class="keyword">for</span> (String url : urlSet) {</span><br><span class="line"> System.out.println(url);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<h1 id="第一行"><a href="#第一行" class="headerlink" title="第一行"></a>第一行</h1><p>逐行来解析下，第一行就是从上下文中获取 <code>RequestMappingHandlerMapping</code> 这个 bean，</p>
|
|
<h1 id="第二行"><a href="#第二行" class="headerlink" title="第二行"></a>第二行</h1><p>然后调用了 <code>getHandlerMethods</code>,<br>这里面具体执行了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Map<T, HandlerMethod> <span class="title function_">getHandlerMethods</span><span class="params">()</span> {</span><br><span class="line"> <span class="built_in">this</span>.mappingRegistry.acquireReadLock();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">return</span> Collections.unmodifiableMap(<span class="built_in">this</span>.mappingRegistry.getMappings());</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">finally</span> {</span><br><span class="line"> <span class="built_in">this</span>.mappingRegistry.releaseReadLock();</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>前后加了锁，重要的就是从 <code>mappingRegistry</code> 中获取 <code>mappings</code>, 这里获取的就是<br><code>org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.MappingRegistry#getMappings</code> 具体的代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Map<T, HandlerMethod> <span class="title function_">getMappings</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.mappingLookup;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>而这个 mappingLookup 再来看下</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">MappingRegistry</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> Map<T, MappingRegistration<T>> registry = <span class="keyword">new</span> <span class="title class_">HashMap</span><>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> Map<T, HandlerMethod> mappingLookup = <span class="keyword">new</span> <span class="title class_">LinkedHashMap</span><>();</span><br></pre></td></tr></table></figure>
|
|
<p>可以看到就是在 MappingRegistry 中的一个变量，至于这个变量是怎么来的，简单的考虑下 springboot 处理请求的流程，就是从 Mapping 去找到对应的 Handler，所以就需要提前将这个对应关系存到这个变量里，<br>具体可以看这 <code>org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.MappingRegistry#register</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">register</span><span class="params">(T mapping, Object handler, Method method)</span> {</span><br><span class="line"> <span class="comment">// Assert that the handler method is not a suspending one.</span></span><br><span class="line"> <span class="keyword">if</span> (KotlinDetector.isKotlinType(method.getDeclaringClass())) {</span><br><span class="line"> Class<?>[] parameterTypes = method.getParameterTypes();</span><br><span class="line"> <span class="keyword">if</span> ((parameterTypes.length > <span class="number">0</span>) && <span class="string">"kotlin.coroutines.Continuation"</span>.equals(parameterTypes[parameterTypes.length - <span class="number">1</span>].getName())) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalStateException</span>(<span class="string">"Unsupported suspending handler method detected: "</span> + method);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">this</span>.readWriteLock.writeLock().lock();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">HandlerMethod</span> <span class="variable">handlerMethod</span> <span class="operator">=</span> createHandlerMethod(handler, method);</span><br><span class="line"> validateMethodMapping(handlerMethod, mapping);</span><br><span class="line"> <span class="built_in">this</span>.mappingLookup.put(mapping, handlerMethod); </span><br></pre></td></tr></table></figure>
|
|
<p>就是在这里会把 mapping 和 handleMethod 对应关系存进去</p>
|
|
<h1 id="第三行"><a href="#第三行" class="headerlink" title="第三行"></a>第三行</h1><p>这里拿的是上面的 <code>map</code> 里的 <code>key</code>，也就是 <code>RequestMappingInfo</code> 也就是 <code>org.springframework.web.servlet.mvc.method.RequestMappingInfo</code><br>而真的 url 就存在 <code>org.springframework.web.servlet.mvc.condition.PatternsRequestCondition</code><br>最终这里面的patterns就是我们要的路径</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">PatternsRequestCondition</span> <span class="keyword">extends</span> <span class="title class_">AbstractRequestCondition</span><PatternsRequestCondition> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="keyword">static</span> Set<String> EMPTY_PATH_PATTERN = Collections.singleton(<span class="string">""</span>);</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> Set<String> patterns;</span><br></pre></td></tr></table></figure>
|
|
<p>写到这下一篇是不是可以介绍下 mapping 的具体注册逻辑</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>SpringBoot</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>springboot 请求响应处理流程</title>
|
|
<url>/2023/08/27/springboot-%E8%AF%B7%E6%B1%82%E5%93%8D%E5%BA%94%E5%A4%84%E7%90%86%E6%B5%81%E7%A8%8B/</url>
|
|
<content><![CDATA[<p>Tomcat 会把请求委托到<br><code>org.springframework.web.servlet.DispatcherServlet#doService</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">doService</span><span class="params">(HttpServletRequest request, HttpServletResponse response)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> logRequest(request);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 省略前面的代码</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> doDispatch(request, response);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后就是调用<br><code>org.springframework.web.servlet.DispatcherServlet#doDispatch</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">doDispatch</span><span class="params">(HttpServletRequest request, HttpServletResponse response)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">ModelAndView</span> <span class="variable">mv</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="type">Exception</span> <span class="variable">dispatchException</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> processedRequest = checkMultipart(request);</span><br><span class="line"> multipartRequestParsed = (processedRequest != request);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 去获取处理器</span></span><br><span class="line"> <span class="comment">// Determine handler for the current request.</span></span><br><span class="line"> mappedHandler = getHandler(processedRequest);</span><br><span class="line"> <span class="keyword">if</span> (mappedHandler == <span class="literal">null</span>) {</span><br><span class="line"> noHandlerFound(processedRequest, response);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Determine handler adapter for the current request.</span></span><br><span class="line"> <span class="type">HandlerAdapter</span> <span class="variable">ha</span> <span class="operator">=</span> getHandlerAdapter(mappedHandler.getHandler());</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Process last-modified header, if supported by the handler.</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">method</span> <span class="operator">=</span> request.getMethod();</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">isGet</span> <span class="operator">=</span> HttpMethod.GET.matches(method);</span><br><span class="line"> <span class="keyword">if</span> (isGet || HttpMethod.HEAD.matches(method)) {</span><br><span class="line"> <span class="type">long</span> <span class="variable">lastModified</span> <span class="operator">=</span> ha.getLastModified(request, mappedHandler.getHandler());</span><br><span class="line"> <span class="keyword">if</span> (<span class="keyword">new</span> <span class="title class_">ServletWebRequest</span>(request, response).checkNotModified(lastModified) && isGet) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!mappedHandler.applyPreHandle(processedRequest, response)) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Actually invoke the handler.</span></span><br><span class="line"> mv = ha.handle(processedRequest, response, mappedHandler.getHandler());</span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
<p>看下这里的逻辑</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Nullable</span></span><br><span class="line"><span class="keyword">protected</span> HandlerExecutionChain <span class="title function_">getHandler</span><span class="params">(HttpServletRequest request)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.handlerMappings != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">for</span> (HandlerMapping mapping : <span class="built_in">this</span>.handlerMappings) {</span><br><span class="line"> <span class="type">HandlerExecutionChain</span> <span class="variable">handler</span> <span class="operator">=</span> mapping.getHandler(request);</span><br><span class="line"> <span class="keyword">if</span> (handler != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> handler;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>可以看到有这些 HandlerMapping<br><img data-src="https://img.nicksxs.me/blog/xqv27E.png"><br>而这里面就是前面提到过的<br><code>org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping</code><br>从这就能找到具体的 Handler<br><code>com.nicksxs.spbdemo.controller.DemoController#test()</code><br>这就是我简单的示例代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@RequestMapping(value = "/test", method = RequestMethod.GET)</span></span><br><span class="line"><span class="meta">@ResponseBody</span></span><br><span class="line"><span class="keyword">public</span> DemoResponse <span class="title function_">test</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">item</span> <span class="operator">=</span> <span class="string">"{\"id\": 1, \"name\": \"nick\"}"</span>;</span><br><span class="line"> <span class="type">ParserConfig</span> <span class="variable">parserConfig</span> <span class="operator">=</span> ParserConfig.getGlobalInstance();</span><br><span class="line"> parserConfig.propertyNamingStrategy = PropertyNamingStrategy.SnakeCase;</span><br><span class="line"> <span class="type">DemoResponse</span> <span class="variable">response</span> <span class="operator">=</span> JSON.parseObject(item, DemoResponse.class, parserConfig);</span><br><span class="line"> <span class="keyword">return</span> response;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>下一步是再获取处理器的适配器，<br><code>org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> HandlerAdapter <span class="title function_">getHandlerAdapter</span><span class="params">(Object handler)</span> <span class="keyword">throws</span> ServletException {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.handlerAdapters != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">for</span> (HandlerAdapter adapter : <span class="built_in">this</span>.handlerAdapters) {</span><br><span class="line"> <span class="keyword">if</span> (adapter.supports(handler)) {</span><br><span class="line"> <span class="keyword">return</span> adapter;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">ServletException</span>(<span class="string">"No adapter for handler ["</span> + handler +</span><br><span class="line"> <span class="string">"]: The DispatcherServlet configuration needs to include a HandlerAdapter that supports this handler"</span>);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>正好这个适配器是调用的父类的 supports 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">final</span> <span class="type">boolean</span> <span class="title function_">supports</span><span class="params">(Object handler)</span> {</span><br><span class="line"> <span class="keyword">return</span> (handler <span class="keyword">instanceof</span> HandlerMethod && supportsInternal((HandlerMethod) handler));</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>而我这个<code>com.nicksxs.spbdemo.controller.DemoController#test()</code><br>就是个包装好的 <code>HandlerMethod</code><br>然后就是调用 <code>ha</code> 的 <code>handle</code> 方法，也是通过模板方法，实际调用的是<br><code>org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter#handle</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="meta">@Nullable</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">final</span> ModelAndView <span class="title function_">handle</span><span class="params">(HttpServletRequest request, HttpServletResponse response, Object handler)</span></span><br><span class="line"> <span class="keyword">throws</span> Exception {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> handleInternal(request, response, (HandlerMethod) handler);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后调用<br><code>org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter#handleInternal</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">protected</span> ModelAndView <span class="title function_">handleInternal</span><span class="params">(HttpServletRequest request,</span></span><br><span class="line"><span class="params"> HttpServletResponse response, HandlerMethod handlerMethod)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"></span><br><span class="line"> ModelAndView mav;</span><br><span class="line"> checkRequest(request);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Execute invokeHandlerMethod in synchronized block if required.</span></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.synchronizeOnSession) {</span><br><span class="line"> <span class="type">HttpSession</span> <span class="variable">session</span> <span class="operator">=</span> request.getSession(<span class="literal">false</span>);</span><br><span class="line"> <span class="keyword">if</span> (session != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">Object</span> <span class="variable">mutex</span> <span class="operator">=</span> WebUtils.getSessionMutex(session);</span><br><span class="line"> <span class="keyword">synchronized</span> (mutex) {</span><br><span class="line"> mav = invokeHandlerMethod(request, response, handlerMethod);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// No HttpSession available -> no mutex necessary</span></span><br><span class="line"> mav = invokeHandlerMethod(request, response, handlerMethod);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// 是否要锁定 session，否则走到这</span></span><br><span class="line"> <span class="comment">// No synchronization on session demanded at all...</span></span><br><span class="line"> mav = invokeHandlerMethod(request, response, handlerMethod);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>继续调用<br><code>org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter#invokeHandlerMethod</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Nullable</span></span><br><span class="line"> <span class="keyword">protected</span> ModelAndView <span class="title function_">invokeHandlerMethod</span><span class="params">(HttpServletRequest request,</span></span><br><span class="line"><span class="params"> HttpServletResponse response, HandlerMethod handlerMethod)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"></span><br><span class="line"> <span class="type">ServletWebRequest</span> <span class="variable">webRequest</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ServletWebRequest</span>(request, response);</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">WebDataBinderFactory</span> <span class="variable">binderFactory</span> <span class="operator">=</span> getDataBinderFactory(handlerMethod);</span><br><span class="line"> <span class="type">ModelFactory</span> <span class="variable">modelFactory</span> <span class="operator">=</span> getModelFactory(handlerMethod, binderFactory);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 包装调用方法，</span></span><br><span class="line"> <span class="type">ServletInvocableHandlerMethod</span> <span class="variable">invocableMethod</span> <span class="operator">=</span> createInvocableHandlerMethod(handlerMethod);</span><br><span class="line"> <span class="comment">// 省略一部分非本次关注核心代码</span></span><br><span class="line"> <span class="comment">// 然后是调用到这</span></span><br><span class="line"> invocableMethod.invokeAndHandle(webRequest, mavContainer);</span><br><span class="line"> <span class="keyword">if</span> (asyncManager.isConcurrentHandlingStarted()) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> getModelAndView(mavContainer, modelFactory, webRequest);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">finally</span> {</span><br><span class="line"> webRequest.requestCompleted();</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>稍微再看一眼<br>第一步是<br><code>org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter#createInvocableHandlerMethod</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> ServletInvocableHandlerMethod <span class="title function_">createInvocableHandlerMethod</span><span class="params">(HandlerMethod handlerMethod)</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">ServletInvocableHandlerMethod</span>(handlerMethod);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>第二步是<br><code>org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod#ServletInvocableHandlerMethod(org.springframework.web.method.HandlerMethod)</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"> <span class="keyword">public</span> <span class="title function_">ServletInvocableHandlerMethod</span><span class="params">(HandlerMethod handlerMethod)</span> {</span><br><span class="line"> <span class="built_in">super</span>(handlerMethod);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>第三步是<br><code>org.springframework.web.method.support.InvocableHandlerMethod#InvocableHandlerMethod(org.springframework.web.method.HandlerMethod)</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"> <span class="keyword">public</span> <span class="title function_">InvocableHandlerMethod</span><span class="params">(HandlerMethod handlerMethod)</span> {</span><br><span class="line"> <span class="built_in">super</span>(handlerMethod);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>第四步是<br><code>org.springframework.web.method.HandlerMethod#HandlerMethod(org.springframework.web.method.HandlerMethod)</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="title function_">HandlerMethod</span><span class="params">(HandlerMethod handlerMethod)</span> {</span><br><span class="line"> Assert.notNull(handlerMethod, <span class="string">"HandlerMethod is required"</span>);</span><br><span class="line"> <span class="built_in">this</span>.bean = handlerMethod.bean;</span><br><span class="line"> <span class="built_in">this</span>.beanFactory = handlerMethod.beanFactory;</span><br><span class="line"> <span class="built_in">this</span>.beanType = handlerMethod.beanType;</span><br><span class="line"> <span class="built_in">this</span>.method = handlerMethod.method;</span><br><span class="line"> <span class="built_in">this</span>.bridgedMethod = handlerMethod.bridgedMethod;</span><br><span class="line"> <span class="built_in">this</span>.parameters = handlerMethod.parameters;</span><br><span class="line"> <span class="built_in">this</span>.responseStatus = handlerMethod.responseStatus;</span><br><span class="line"> <span class="built_in">this</span>.responseStatusReason = handlerMethod.responseStatusReason;</span><br><span class="line"> <span class="built_in">this</span>.description = handlerMethod.description;</span><br><span class="line"> <span class="built_in">this</span>.resolvedFromHandlerMethod = handlerMethod.resolvedFromHandlerMethod;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这是个继承关系，一直调用到最顶层的父类的构造方法，其实就是拷贝，然后继续调用<br><code>org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod#invokeAndHandle</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">invokeAndHandle</span><span class="params">(ServletWebRequest webRequest, ModelAndViewContainer mavContainer,</span></span><br><span class="line"><span class="params"> Object... providedArgs)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 调用请求</span></span><br><span class="line"> <span class="type">Object</span> <span class="variable">returnValue</span> <span class="operator">=</span> invokeForRequest(webRequest, mavContainer, providedArgs);</span><br><span class="line"> <span class="comment">// 稍微忽略下后面的代码</span></span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>继续调用<br><code>org.springframework.web.method.support.InvocableHandlerMethod#invokeForRequest</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Nullable</span></span><br><span class="line"><span class="keyword">public</span> Object <span class="title function_">invokeForRequest</span><span class="params">(NativeWebRequest request, <span class="meta">@Nullable</span> ModelAndViewContainer mavContainer,</span></span><br><span class="line"><span class="params"> Object... providedArgs)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"></span><br><span class="line"> Object[] args = getMethodArgumentValues(request, mavContainer, providedArgs);</span><br><span class="line"> <span class="keyword">if</span> (logger.isTraceEnabled()) {</span><br><span class="line"> logger.trace(<span class="string">"Arguments: "</span> + Arrays.toString(args));</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> doInvoke(args);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>来到了最核心处<br> <code>org.springframework.web.method.support.InvocableHandlerMethod#doInvoke</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Nullable</span></span><br><span class="line"><span class="keyword">protected</span> Object <span class="title function_">doInvoke</span><span class="params">(Object... args)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="type">Method</span> <span class="variable">method</span> <span class="operator">=</span> getBridgedMethod();</span><br><span class="line"> ReflectionUtils.makeAccessible(method);</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (KotlinDetector.isSuspendingFunction(method)) {</span><br><span class="line"> <span class="keyword">return</span> CoroutinesUtils.invokeSuspendingFunction(method, getBean(), args);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 会走到这里，获取到 bean，而这个 bean 就是前面构造方法里赋值的，最开始被放在 handler 里面，然后调用方法</span></span><br><span class="line"> <span class="keyword">return</span> method.invoke(getBean(), args);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>SpringBoot</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>ssh 小技巧-端口转发</title>
|
|
<url>/2023/03/26/ssh-%E5%B0%8F%E6%8A%80%E5%B7%A7-%E7%AB%AF%E5%8F%A3%E8%BD%AC%E5%8F%91/</url>
|
|
<content><![CDATA[<p>我们在使用 ssh 连接的使用有一个很好用功能，就是端口转发，而且使用的方式也很多样，比如我们经常用 vscode 来做远程开发的话，一般远程连接就可以基于 ssh，前面也介绍过 vscode 的端口转发，并且可以配置到 .ssh/config 配置文件里，只不过最近在一次使用的过程中发现了一个问题，就是在一台 Ubuntu 的某云服务器上想 ssh 到另一台服务器上，并且做下端口映射，但是发现报了个错，</p>
|
|
<figure class="highlight sh"><table><tr><td class="code"><pre><span class="line"><span class="built_in">bind</span>: Cannot assign requested address</span><br></pre></td></tr></table></figure>
|
|
<p>查了下这个问题，猜测是不是端口已经被占用了，查了下并不是，然后想到是不是端口是系统保留的，</p>
|
|
<figure class="highlight sh"><table><tr><td class="code"><pre><span class="line">sysctl -a |grep port_range</span><br></pre></td></tr></table></figure>
|
|
<p>结果中</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">net.ipv4.ip_local_port_range = 50000 65000 -----意味着50000~65000端口可用</span><br></pre></td></tr></table></figure>
|
|
<p>发现也不是，没有限制，最后才查到这个原因是默认如果有 ipv6 的话会使用 ipv6 的地址做映射<br>所以如果是命令连接做端口转发的话，</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">ssh -4 -L 11234:localhost:1234 x.x.x.x</span><br></pre></td></tr></table></figure>
|
|
<p>使用<code>-4</code>来制定通过 ipv4 地址来做映射<br>如果是在 <code>.ssh/config</code> 中配置的话可以直接指定所有的连接都走 ipv4</p>
|
|
<figure class="highlight sh"><table><tr><td class="code"><pre><span class="line">Host *</span><br><span class="line"> AddressFamily inet</span><br></pre></td></tr></table></figure>
|
|
<p><code>inet</code>代表 ipv4，<code>inet6</code>代表 ipv6<br>AddressFamily 的所有取值范围是：”any”(默认)、”inet”(仅IPv4)、”inet6”(仅IPv6)。<br>另外此类问题还可以通过 <code>ssh -v</code> 来打印更具体的信息</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>ssh</category>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>ssh</tag>
|
|
<tag>端口转发</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>summary-ranges-228</title>
|
|
<url>/2016/10/12/summary-ranges-228/</url>
|
|
<content><![CDATA[<h4 id="problem"><a href="#problem" class="headerlink" title="problem"></a>problem</h4><p>Given a sorted integer array without duplicates, return the summary of its ranges.</p>
|
|
<p>For example, given <code>[0,1,2,4,5,7]</code>, return <code>["0->2","4->5","7"]</code>.</p>
|
|
<h4 id="题解"><a href="#题解" class="headerlink" title="题解"></a>题解</h4><p>每一个区间的起点<code>nums[i]</code>加上<code>j</code>是否等于<code>nums[i+j]</code><br><a href="http://www.cnblogs.com/grandyang/p/4603555.html">参考</a></p>
|
|
<h4 id="Code"><a href="#Code" class="headerlink" title="Code"></a>Code</h4><figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">Solution</span> {</span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line"> <span class="function">vector<string> <span class="title">summaryRanges</span><span class="params">(vector<<span class="type">int</span>>& nums)</span> </span>{</span><br><span class="line"> <span class="type">int</span> i = <span class="number">0</span>, j = <span class="number">1</span>, n;</span><br><span class="line"> vector<string> res;</span><br><span class="line"> n = nums.<span class="built_in">size</span>();</span><br><span class="line"> <span class="keyword">while</span>(i < n){</span><br><span class="line"> j = <span class="number">1</span>;</span><br><span class="line"> <span class="keyword">while</span>(j < n && nums[i+j] - nums[i] == j) j++;</span><br><span class="line"> res.<span class="built_in">push_back</span>(j <= <span class="number">1</span> ? <span class="built_in">to_string</span>(nums[i]) : <span class="built_in">to_string</span>(nums[i]) + <span class="string">"->"</span> + <span class="built_in">to_string</span>(nums[i + j - <span class="number">1</span>]));</span><br><span class="line"> i += j;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> res;</span><br><span class="line"> }</span><br><span class="line">};</span><br></pre></td></tr></table></figure>]]></content>
|
|
<categories>
|
|
<category>leetcode</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>leetcode</tag>
|
|
<tag>c++</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>swoole-websocket-test</title>
|
|
<url>/2016/07/13/swoole-websocket-test/</url>
|
|
<content><![CDATA[<h3 id="玩一下swoole的websocket"><a href="#玩一下swoole的websocket" class="headerlink" title="玩一下swoole的websocket"></a>玩一下swoole的websocket</h3><p>WebSocket是HTML5开始提供的一种在单个<a href="https://zh.wikipedia.org/wiki/TCP">TCP</a>连接上进行<a href="https://zh.wikipedia.org/wiki/%E5%85%A8%E9%9B%99%E5%B7%A5">全双工</a>通讯的协议。WebSocket通信协议于2011年被<a href="https://zh.wikipedia.org/wiki/Internet_Engineering_Task_Force">IETF</a>定为标准RFC 6455，WebSocketAPI被W3C定为标准。<br>,在web私信，im等应用较多。背景和优缺点可以参看<a href="https://zh.wikipedia.org/wiki/WebSocket">wiki</a>。</p>
|
|
<h3 id="环境准备"><a href="#环境准备" class="headerlink" title="环境准备"></a>环境准备</h3><p>因为swoole官方还不支持windows，所以需要装下linux,之前都是用ubuntu，<br>这次就试一下centos7，还是满好看的，虽然虚拟机会默认最小安装，需要在安装<br>时自己选择带gnome的，当然最小安装也是可以的，只是最后需要改下防火墙。<br>然后是装下PHP，Nginx什么的，我是用<a href="https://oneinstack.com/">Oneinstack</a>，可以按需安装<br>给做这个的大大点个赞。</p>
|
|
<span id="more"></span>
|
|
|
|
<h2 id="swoole"><a href="#swoole" class="headerlink" title="swoole"></a>swoole</h2><p>1.install via pecl</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">pecl install swoole</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>2.install from source</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">sudo apt-get install php5-dev</span><br><span class="line">git clone https://github.com/swoole/swoole-src.git</span><br><span class="line">cd swoole-src</span><br><span class="line">phpize</span><br><span class="line">./configure</span><br><span class="line">make && make install</span><br></pre></td></tr></table></figure>
|
|
<p>3.add extension</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">extension = swoole.so</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>4.test extension</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">php -m | grep swoole</span><br></pre></td></tr></table></figure>
|
|
<p>如果存在就代表安装成功啦</p>
|
|
<h2 id="Exec"><a href="#Exec" class="headerlink" title="Exec"></a>Exec</h2><p>实现代码看了这位仁兄的<a href="http://www.jianshu.com/p/fedbb9d2d999">代码</a></p>
|
|
<p>还是贴一下代码<br>服务端：</p>
|
|
<figure class="highlight php"><table><tr><td class="code"><pre><span class="line"><span class="comment">//创建websocket服务器对象，监听0.0.0.0:9502端口</span></span><br><span class="line"><span class="variable">$ws</span> = <span class="keyword">new</span> <span class="title function_ invoke__">swoole_websocket_server</span>(<span class="string">"0.0.0.0"</span>, <span class="number">9502</span>);</span><br><span class="line"></span><br><span class="line"><span class="comment">//监听WebSocket连接打开事件</span></span><br><span class="line"><span class="variable">$ws</span>-><span class="title function_ invoke__">on</span>(<span class="string">'open'</span>, function (<span class="variable">$ws</span>, <span class="variable">$request</span>) {</span><br><span class="line"> <span class="variable">$fd</span>[] = <span class="variable">$request</span>->fd;</span><br><span class="line"> <span class="variable">$GLOBALS</span>[<span class="string">'fd'</span>][] = <span class="variable">$fd</span>;</span><br><span class="line"> <span class="comment">//区别下当前用户</span></span><br><span class="line"> <span class="variable">$ws</span>-><span class="title function_ invoke__">push</span>(<span class="variable">$request</span>->fd, <span class="string">"hello user<span class="subst">{$request->fd}</span>, welcome\n"</span>); </span><br><span class="line">});</span><br><span class="line"></span><br><span class="line"><span class="comment">//监听WebSocket消息事件</span></span><br><span class="line"><span class="variable">$ws</span>-><span class="title function_ invoke__">on</span>(<span class="string">'message'</span>, function (<span class="variable">$ws</span>, <span class="variable">$frame</span>) {</span><br><span class="line"> <span class="variable">$msg</span> = <span class="string">'from'</span>.<span class="variable">$frame</span>->fd.<span class="string">":<span class="subst">{$frame->data}</span>\n"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">foreach</span>(<span class="variable">$GLOBALS</span>[<span class="string">'fd'</span>] <span class="keyword">as</span> <span class="variable">$aa</span>){</span><br><span class="line"> <span class="keyword">foreach</span>(<span class="variable">$aa</span> <span class="keyword">as</span> <span class="variable">$i</span>){</span><br><span class="line"> <span class="keyword">if</span>(<span class="variable">$i</span> != <span class="variable">$frame</span>->fd) {</span><br><span class="line"> <span class="comment"># code...</span></span><br><span class="line"> <span class="variable">$ws</span>-><span class="title function_ invoke__">push</span>(<span class="variable">$i</span>,<span class="variable">$msg</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">});</span><br><span class="line"></span><br><span class="line"><span class="comment">//监听WebSocket连接关闭事件</span></span><br><span class="line"><span class="variable">$ws</span>-><span class="title function_ invoke__">on</span>(<span class="string">'close'</span>, function (<span class="variable">$ws</span>, <span class="variable">$fd</span>) {</span><br><span class="line"> <span class="keyword">echo</span> <span class="string">"client-<span class="subst">{$fd}</span> is closed\n"</span>;</span><br><span class="line">});</span><br><span class="line"></span><br><span class="line"><span class="variable">$ws</span>-><span class="title function_ invoke__">start</span>();</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>客户端：</p>
|
|
<figure class="highlight html"><table><tr><td class="code"><pre><span class="line"><span class="meta"><!DOCTYPE <span class="keyword">html</span>></span></span><br><span class="line"><span class="tag"><<span class="name">html</span> <span class="attr">lang</span>=<span class="string">"en"</span>></span></span><br><span class="line"><span class="tag"><<span class="name">head</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">meta</span> <span class="attr">charset</span>=<span class="string">"UTF-8"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">title</span>></span>Title<span class="tag"></<span class="name">title</span>></span></span><br><span class="line"><span class="tag"></<span class="name">head</span>></span></span><br><span class="line"><span class="tag"><<span class="name">body</span>></span></span><br><span class="line"><span class="tag"><<span class="name">div</span> <span class="attr">id</span>=<span class="string">"msg"</span>></span><span class="tag"></<span class="name">div</span>></span></span><br><span class="line"><span class="tag"><<span class="name">input</span> <span class="attr">type</span>=<span class="string">"text"</span> <span class="attr">id</span>=<span class="string">"text"</span>></span></span><br><span class="line"><span class="tag"><<span class="name">input</span> <span class="attr">type</span>=<span class="string">"submit"</span> <span class="attr">value</span>=<span class="string">"发送数据"</span> <span class="attr">onclick</span>=<span class="string">"song()"</span>></span></span><br><span class="line"><span class="tag"></<span class="name">body</span>></span></span><br><span class="line"><span class="tag"><<span class="name">script</span>></span><span class="language-javascript"></span></span><br><span class="line"><span class="language-javascript"> <span class="keyword">var</span> msg = <span class="variable language_">document</span>.<span class="title function_">getElementById</span>(<span class="string">"msg"</span>);</span></span><br><span class="line"><span class="language-javascript"> <span class="keyword">var</span> wsServer = <span class="string">'ws://0.0.0.0:9502'</span>;</span></span><br><span class="line"><span class="language-javascript"> <span class="comment">//调用websocket对象建立连接：</span></span></span><br><span class="line"><span class="language-javascript"> <span class="comment">//参数：ws/wss(加密)：//ip:port （字符串）</span></span></span><br><span class="line"><span class="language-javascript"> <span class="keyword">var</span> websocket = <span class="keyword">new</span> <span class="title class_">WebSocket</span>(wsServer);</span></span><br><span class="line"><span class="language-javascript"> <span class="comment">//onopen监听连接打开</span></span></span><br><span class="line"><span class="language-javascript"> websocket.<span class="property">onopen</span> = <span class="keyword">function</span> (<span class="params">evt</span>) {</span></span><br><span class="line"><span class="language-javascript"> <span class="comment">//websocket.readyState 属性：</span></span></span><br><span class="line"><span class="language-javascript"> <span class="comment">/*</span></span></span><br><span class="line"><span class="comment"><span class="language-javascript"> CONNECTING 0 The connection is not yet open.</span></span></span><br><span class="line"><span class="comment"><span class="language-javascript"> OPEN 1 The connection is open and ready to communicate.</span></span></span><br><span class="line"><span class="comment"><span class="language-javascript"> CLOSING 2 The connection is in the process of closing.</span></span></span><br><span class="line"><span class="comment"><span class="language-javascript"> CLOSED 3 The connection is closed or couldn't be opened.</span></span></span><br><span class="line"><span class="comment"><span class="language-javascript"> */</span></span></span><br><span class="line"><span class="language-javascript"> msg.<span class="property">innerHTML</span> = websocket.<span class="property">readyState</span>;</span></span><br><span class="line"><span class="language-javascript"> };</span></span><br><span class="line"><span class="language-javascript"></span></span><br><span class="line"><span class="language-javascript"> <span class="keyword">function</span> <span class="title function_">song</span>(<span class="params"></span>){</span></span><br><span class="line"><span class="language-javascript"> <span class="keyword">var</span> text = <span class="variable language_">document</span>.<span class="title function_">getElementById</span>(<span class="string">'text'</span>).<span class="property">value</span>;</span></span><br><span class="line"><span class="language-javascript"> <span class="variable language_">document</span>.<span class="title function_">getElementById</span>(<span class="string">'text'</span>).<span class="property">value</span> = <span class="string">''</span>;</span></span><br><span class="line"><span class="language-javascript"> <span class="comment">//向服务器发送数据</span></span></span><br><span class="line"><span class="language-javascript"> websocket.<span class="title function_">send</span>(text);</span></span><br><span class="line"><span class="language-javascript"> }</span></span><br><span class="line"><span class="language-javascript"> <span class="comment">//监听连接关闭</span></span></span><br><span class="line"><span class="language-javascript"><span class="comment">// websocket.onclose = function (evt) {</span></span></span><br><span class="line"><span class="language-javascript"><span class="comment">// console.log("Disconnected");</span></span></span><br><span class="line"><span class="language-javascript"><span class="comment">// };</span></span></span><br><span class="line"><span class="language-javascript"></span></span><br><span class="line"><span class="language-javascript"> <span class="comment">//onmessage 监听服务器数据推送</span></span></span><br><span class="line"><span class="language-javascript"> websocket.<span class="property">onmessage</span> = <span class="keyword">function</span> (<span class="params">evt</span>) {</span></span><br><span class="line"><span class="language-javascript"> msg.<span class="property">innerHTML</span> += evt.<span class="property">data</span> +<span class="string">'<br>'</span>;</span></span><br><span class="line"><span class="language-javascript"><span class="comment">// console.log('Retrieved data from server: ' + evt.data);</span></span></span><br><span class="line"><span class="language-javascript"> };</span></span><br><span class="line"><span class="language-javascript"><span class="comment">//监听连接错误信息</span></span></span><br><span class="line"><span class="language-javascript"><span class="comment">// websocket.onerror = function (evt, e) {</span></span></span><br><span class="line"><span class="language-javascript"><span class="comment">// console.log('Error occured: ' + evt.data);</span></span></span><br><span class="line"><span class="language-javascript"><span class="comment">// };</span></span></span><br><span class="line"><span class="language-javascript"></span></span><br><span class="line"><span class="language-javascript"></span><span class="tag"></<span class="name">script</span>></span></span><br><span class="line"><span class="tag"></<span class="name">html</span>></span></span><br></pre></td></tr></table></figure>
|
|
|
|
<p>做了个循环，将当前用户的消息发送给同时在线的其他用户，比较简陋，如下图<br>user1:<br><img data-src="https://ooo.0o0.ooo/2016/07/13/578665c09066f.png" alt="NH}()5}1DTLTKZ%HUQ`4L(V.png](https://ooo.0o0.ooo/2016/07/13/578665c07d94c.png)
|
|
user2:
|
|
|
|
user3:
|
|
![QK8EU5`9TQNYIG_4YFU@DJN.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>php</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>websocket</tag>
|
|
<tag>swoole</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>win 下 vmware 虚拟机搭建黑裙 nas 的小思路</title>
|
|
<url>/2023/06/04/win-%E4%B8%8B-vmware-%E8%99%9A%E6%8B%9F%E6%9C%BA%E6%90%AD%E5%BB%BA%E9%BB%91%E8%A3%99-nas-%E7%9A%84%E5%B0%8F%E6%80%9D%E8%B7%AF/</url>
|
|
<content><![CDATA[<p>上次说 nas 的方案我是在 win10 下使用vmware workstation 搭建的黑裙虚拟机，采用 sata 物理磁盘直通的方式，算是跑通了黑裙的基础使用模式，但是后来发现的一个问题是之前没考虑到的，我买了不带 f 的处理器就是为了核显能做硬解，但是因为 cpu 是通过 vmware 虚拟的，目前看来是没法直通核显的，我是使用的 jellyfin 套件，一开始使用是默认的刮削方式，而且把电视剧当成了电影在刮削，所以基本不能看，后面使用了 tmm 作为刮削工具，可以手动填写 imdb 的id 来进行搜索，一般比较正式的剧都可以在豆瓣上找到的，然后让 jellyfin 只作为媒体管理器，但是前面的问题还是没解决，所以考虑了下可以在win10 下直接运行 jellyfin，媒体目录使用挂载在 nas 里的盘，这样 jellyfin 就能直接调用核显了，也算是把 win10 本身给利用起来了，并且文件的管理还是在黑裙中。<br>现在想来其实我这个方案还是不太合理，cpu 性能有点过剩想通过虚拟机的形式进行隔离使用，但是购买带核显的 cpu 最大的目的却没有实现，如果是直接裸机部署黑裙的话，真的是觉得 cpu 有点太浪费了，毕竟 passmark评分有 1w3 的cpu，只用来跑黑裙，所以网上的很多建议也是合理的，不过我可能是 win10 用的比较多了，还是习惯有 win 的环境。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>nas</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>nas</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>wordpress 忘记密码的一种解决方法</title>
|
|
<url>/2021/12/05/wordpress-%E5%BF%98%E8%AE%B0%E5%AF%86%E7%A0%81%E7%9A%84%E4%B8%80%E7%A7%8D%E8%A7%A3%E5%86%B3%E6%96%B9%E6%B3%95/</url>
|
|
<content><![CDATA[<p>前阵子搭了个 WordPress，但是没怎么用，前两天发现忘了登录密码了，最近不知道是什么情况，chrome 的记住密码跟历史记录感觉有点问题，历史记录丢了不少东西，可能是时间太久了，但是理论上应该有 LRU 这种策略的，有些还比较常用，还有记住密码，因为个人域名都是用子域名分配给各个服务，有些记住了，有些又没记住密码，略蛋疼，所以就找了下这个方案。<br>当然这个方案不是最优的，有很多限制，首先就是要能够登陆 WordPress 的数据库，不然这个方法是没用的。<br>首先不管用什么方式（别违法）先登陆数据库，选择 WordPress 的数据库，可以看到里面有几个表，我们的目标就是 <code>wp_users</code> 表，用 <code>select</code> 查询看下可以看到有用户的数据，如果是像我这样搭着玩的没有创建其他用户的话应该就只有一个用户，那我们的表里的用户数据就只会有一条，当然多条的话可以通过用户名来找<br><img data-src="https://img.nicksxs.me/uPic/9xEy8t.png"><br>然后可能我这个版本是这样，没有装额外的插件，密码只是经过了 MD5 的单向哈希，所以我们可以设定一个新密码，然后用 MD5 编码后直接更新进去</p>
|
|
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="keyword">UPDATE</span> wp_users <span class="keyword">SET</span> user_pass <span class="operator">=</span> MD5(<span class="string">'123456'</span>) <span class="keyword">WHERE</span> ID <span class="operator">=</span> <span class="number">1</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后就能用自己的账户跟刚才更新的密码登录了。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>小技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>WordPress</tag>
|
|
<tag>小技巧</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>《寻羊历险记》读后感</title>
|
|
<url>/2023/07/23/%E3%80%8A%E5%AF%BB%E7%BE%8A%E5%8E%86%E9%99%A9%E8%AE%B0%E3%80%8B%E8%AF%BB%E5%90%8E%E6%84%9F/</url>
|
|
<content><![CDATA[<p> 最近本来是在读《舞舞舞》，然后看到有介绍说，这个跟《寻羊历险记》是有情节上的关联，所以就先去看了《寻羊历险记》，《寻羊历险记》也是村上春树的第一本成规模的长篇小说，也可以认为是《舞舞舞》的前篇。<br> 最开始这个情节跟之前的刺杀骑士团长还是哪本有点类似，都有跟老婆离婚了，主角应该是个跟朋友一起开翻译社的，后面也开始做广告相关的，做到了经济收益还不错的阶段，也有一些挺哲学的对话，朋友觉得这么赚钱不地道（可能也是觉得这样忘了初心），在离婚以后又结交了一个耳朵很好看的女友，这个女友也是个比较抽象的存在，描述中给人感觉是一个外貌很普通的女孩，但是耳朵漂亮的惊为天人，不知道是不是有什么隐喻，感觉现实中没见过这样的人，女友平时把耳朵遮起来不轻易露出来，只有在跟主角做爱的时候才露出来<br> 主体剧情是因为男主在广告中用了一张一位叫“鼠”的朋友寄给他的一张包含一只特殊的羊的照片，就有个政界大佬的秘书找过来，逼迫主角要找到照片上的羊，这只羊背上有星纹，基本不可能属于在日本当时可能存在的羊的品种，因为这位政界大佬快病危了，所以要求主角必须在一个月内找到这只羊，因为这里把这只神秘的羊塑造成神一样的存在，这位政界大佬在年轻时被这只羊上身，后面就开始在政治事业上大展宏图，就基本成了能左右整个日本走向的巨佬，但近期随着巨佬，身体状态慢慢变差，这只羊就从他身上消失了，所以秘书要主角必须找到这只羊，不然基本会让主角的翻译社完蛋，这样主角就会面临破产赔偿等严重后果，只是说主角本来也一直是这种丧气存在，再说这么茫茫人海找个人都难，还要找这么一只玄乎的不太可能真实存在的羊，所以基本是想要放弃的，结果刚才说的耳朵很漂亮的女友却有着神乎其神的直觉，就觉得能找到，然后他们就踏上了巡羊的旅程，一开始到了札幌，寻找一无所获，然后神奇的是女友推荐一定要住的海豚宾馆，恰恰是一切线索的源头，宾馆老板的父亲正好是羊博士，在年轻的时候被羊上身过，后面离开后就去了那位巨佬身上，让巨佬成了真正控制日本的地下帝国的王，跟羊博士咨询后知道羊可能在十二瀑镇的牧场出现过，所以一路找寻，发现其实这个牧场中有个别墅正好是主角朋友“鼠”的，到了别墅后出现了个神秘的羊男，这个羊男真的不太明白是怎么个存在，就是让主角的女友回去了，然后最后一个肉体承载着“鼠”的灵魂跟主角有了一次接触，主角呆在这个别墅过着百无聊赖的生活，在才到有一次羊男来跟他交流的时候其实是承载着鼠的灵魂，就觉得是不是一切都是在忽悠他，离一个月期限也越来越近了，而在发怒之后，“鼠”真的出现后，但是不是真的“鼠”，而是只有他的灵魂，因为“鼠”已经死了，因为不想被“羊”附身，成为羊壳，并且让主角设定好时钟后的装置后尽快下山，第二天主角离开上了火车后山上牧场就传来爆炸声，“鼠”已经跟羊同归于尽了，免得再有其他人被羊附身，主角也很难过，回到故乡在四周大海已经被填掉了的旧防波堤上大哭悼念逝去的“鼠”。<br> 其实对于没什么时代概念或者对村上一直以来的观念不是特别敏感的，对这本书讲了啥有点云里雾里，后面看了一些简单的解释就是羊其实代表日本帝国主义和资本主义，可能是村上本人反帝国主义，军国主义和资本主义的一个表征，想来也有些道理，不然“鼠”的牺牲就感觉比较没意义，但是另一方面我的理解也可能是对自由的向往的表达，被羊控制，虽然可以成就“伟大”的事业，但是也丧失了自我。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>读后感</category>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>读后感</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>《垃圾回收算法手册读书》笔记之整理算法</title>
|
|
<url>/2021/03/07/%E3%80%8A%E5%9E%83%E5%9C%BE%E5%9B%9E%E6%94%B6%E7%AE%97%E6%B3%95%E6%89%8B%E5%86%8C%E8%AF%BB%E4%B9%A6%E3%80%8B%E7%AC%94%E8%AE%B0%E4%B9%8B%E6%95%B4%E7%90%86%E7%AE%97%E6%B3%95/</url>
|
|
<content><![CDATA[<p>最近看了下这本垃圾回收算法手册，看到了第三章的标记-整理回收算法，做个简单的读书笔记</p>
|
|
<h3 id="双指针整理算法"><a href="#双指针整理算法" class="headerlink" title="双指针整理算法"></a>双指针整理算法</h3><p>对于一块待整理区域，通过两个指针，free 在区域的起始端，scan 指针在区域的末端，free 指针从前往后知道找到空闲区域，scan 从后往前一直找到存活对象，当 free 指针未与 scan 指针交叉时，会给 scan 位置的对象特定位置标记上 free 的地址，即将要转移的地址，不过这里有个限制，这种整理算法一般会用于对象大小统一的情况，否则 free 指针扫描时还需要匹配scan 指针扫描到的存活对象的大小。<br><img data-src="https://img.nicksxs.me/uPic/yLQlgj.png"></p>
|
|
<h3 id="Lisp-2-整理算法"><a href="#Lisp-2-整理算法" class="headerlink" title="Lisp 2 整理算法"></a>Lisp 2 整理算法</h3><p>需要三次完整遍历堆区域<br>第一遍是遍历后将计算出所有对象的最终地址（转发地址）<br>第二遍是使用转发地址更新赋值器线程根以及被标记对象中的引用，该操作将确保它们指向对象的新位置<br>第三次遍历是relocate最终将存活对象移动到其新的目标位置</p>
|
|
<h3 id="引线整理算法"><a href="#引线整理算法" class="headerlink" title="引线整理算法"></a>引线整理算法</h3><p>这个真的长见识了，<br><img data-src="https://img.nicksxs.me/uPic/6yeA7n.png"><br>可以看到，原来是 A，B，C 对象引用了 N，这里会在第一次遍历的时候把这种引用反过来，让 N 的对象头部保存下 A 的地址，表示这类引用，然后在遍历到 B 的时候在链起来，到最后就会把所有引用了 N 对象的所有对象通过引线链起来，在第二次遍历的时候就把更新A，B，C 对象引用的 N 地址，并且移动 N 对象</p>
|
|
<h3 id="单次遍历算法"><a href="#单次遍历算法" class="headerlink" title="单次遍历算法"></a>单次遍历算法</h3><p>这个一直提到过位图的实现方式，<br><img data-src="https://img.nicksxs.me/uPic/Jqtuzu.png"><br>可以看到在第一步会先通过位图标记，标记的方式是位图的每一位对应的堆内存的一个字（这里可能指的是 byte 吧），然后将一个存活对象的内存区域的第一个字跟最后一个字标记，这里如果在通过普通的方式就还需要一个地方在存转发地址，但是因为具体的位置可以通过位图算出来，也就不需要额外记录了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>gc</category>
|
|
<category>jvm</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
<tag>gc</tag>
|
|
<tag>标记整理</tag>
|
|
<tag>垃圾回收</tag>
|
|
<tag>jvm</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>《长安的荔枝》读后感</title>
|
|
<url>/2022/07/17/%E3%80%8A%E9%95%BF%E5%AE%89%E7%9A%84%E8%8D%94%E6%9E%9D%E3%80%8B%E8%AF%BB%E5%90%8E%E6%84%9F/</url>
|
|
<content><![CDATA[<p>断断续续地看完了马伯庸老师的《长安的荔枝》，一开始是看这本书在排行榜排得很高，又是马伯庸的，之前看过他的《古董局中局》，还是很有意思的，而且正好是比较短的，不过前后也拖了蛮久才看完，看完后读了下马老师自己写的后记，就特别有感触。<br>整个故事是围绕一个上林署监事李善德被委任一项给贵妃送荔枝的差事展开，“长安回望绣成堆，山顶千门次第开，一骑红尘妃子笑，无人知是荔枝来”，以前没细究过这个送荔枝的过程，但是以以前的运输速度和保鲜条件，感觉也不是太现实，所以主人公一开始就以为只是像以往一样是送荔枝干这种，能比较方便运输，不容易变质的，结果发现其实是同僚在坑他，这次是要在贵妃生辰的时候给贵妃送来新鲜的岭南荔枝，用比较时兴的词来说，这就是个送命题啊，鲜荔枝一日色变，两日香变，三日味变，同僚的还有杜甫跟韩承，都觉得老李可以直接写休书了，保全家人，不然就是全家送命，李善德也觉得基本算是判刑了，而且其实是这事被转了几次，最后到老李所在的上林署，主管为了骗他接下这个活还特意在文书上把荔枝鲜的“鲜”字贴住，那会叫做“贴黄”，变成了荔枝“煎”，所以说官场险恶，大家都想把这烫手山芋丢出去，结果丢到了我们老实的老李头上，但是从接到这个通知到贵妃的生辰六月初一还有挺长的时间，其实这个活虽然送命，但是在前期这个“荔枝使”也基本就是类似带着尚方宝剑，御赐黄马褂的职位，随便申请经费，不必像常规的部门费用需要定预算，申请后再层层审批，而是特事特批特办的耍赖做法，所以在这段时间是能够潇洒挥霍一下的。其实可以好好地捞一波给妻女，然后写下和离，在自己死后能让她们过的好一些，但最后还是在杜甫的一番劝导下做出了尝试一番的决定，因为也没其他办法，既是退无可退，何不向前拼死一搏，其实说到这，我觉得看这本书感觉有所收获的第一点，有时候总觉得事情没戏了，想躺平放弃了，但是这样其实这个结果是不会变好的，尝试努力，拼尽全力搏一搏，说不定会有所改观，至少不会变更坏了。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>读后感</category>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>读后感</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>一个 nginx 的简单记忆点</title>
|
|
<url>/2022/08/21/%E4%B8%80%E4%B8%AA-nginx-%E7%9A%84%E7%AE%80%E5%8D%95%E8%AE%B0%E5%BF%86%E7%82%B9/</url>
|
|
<content><![CDATA[<p>上周在处理一个 nginx 配置的时候，发现了一个之前不理解的小点，说一个场景，就是我们一般的处理方式就是一个 ip 端口只能配置一个域名的服务，比如 <a href="https://nicksxs.me/">https://nicksxs.me</a> 对应配置到 127.0.0.1:443，如果我想要把 <a href="https://nicksxs.com/">https://nicksxs.com</a> 也解析到这个服务器，并转发到不同的下游，这里就需要借助所谓的 SNI 的功能</p>
|
|
<h3 id="Server-Name-Indication"><a href="#Server-Name-Indication" class="headerlink" title="Server Name Indication"></a>Server Name Indication</h3><p>A more generic solution for running several HTTPS servers on a single IP address is TLS Server Name Indication extension (SNI, RFC 6066), which allows a browser to pass a requested server name during the SSL handshake and, therefore, the server will know which certificate it should use for the connection. SNI is currently supported by most modern browsers, though may not be used by some old or special clients.<br><a href="http://nginx.org/en/docs/http/configuring_https_servers.html#sni">来源</a><br>机翻一下：在单个 IP 地址上运行多个 HTTPS 服务器的更通用的解决方案是 TLS 服务器名称指示扩展（SNI，RFC 6066），它允许浏览器在 SSL 握手期间传递请求的服务器名称，因此，服务器将知道哪个 它应该用于连接的证书。 目前大多数现代浏览器都支持 SNI，但某些旧的或特殊的客户端可能不使用 SNI。</p>
|
|
<p>首先我们需要确认 sni 已被支持<br><img data-src="https://img.nicksxs.me/uPic/VQTZu8.png"><br>在实际的配置中就可以这样</p>
|
|
<figure class="highlight nginx"><table><tr><td class="code"><pre><span class="line"><span class="section">stream</span> {</span><br><span class="line"> <span class="attribute">map</span> <span class="variable">$ssl_preread_server_name</span> <span class="variable">$stream_map</span> {</span><br><span class="line"> nicksxs.<span class="attribute">me</span> nme;</span><br><span class="line"> nicksxs.<span class="attribute">com</span> ncom;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="section">upstream</span> nme {</span><br><span class="line"> <span class="attribute">server</span> <span class="number">127.0.0.1:8000</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="section">upstream</span> ncom {</span><br><span class="line"> <span class="attribute">server</span> <span class="number">127.0.0.1:8001</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="section">server</span> {</span><br><span class="line"> <span class="attribute">listen</span> <span class="number">443</span> reuseport;</span><br><span class="line"> <span class="attribute">proxy_pass</span> <span class="variable">$stream_map</span>;</span><br><span class="line"> <span class="attribute">ssl_preread</span> <span class="literal">on</span>;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>类似这样，但是这个理解是非常肤浅和不完善的，只是简单记忆下，后续再进行补充完整</p>
|
|
<p>还有一点就是我们在配置的时候经常配置就是 server_name，但是会看到直接在使用 ssl_server_name，<br>其实在listen 标识了 ssl, 对应的 ssl_server_name 就等于 server_name，不需要额外处理了。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>nginx</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>nginx</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>一个经典的fastjson反序列化问题记录</title>
|
|
<url>/2024/08/11/%E4%B8%80%E4%B8%AA%E7%BB%8F%E5%85%B8%E7%9A%84fastjson%E5%8F%8D%E5%BA%8F%E5%88%97%E5%8C%96%E9%97%AE%E9%A2%98%E8%AE%B0%E5%BD%95/</url>
|
|
<content><![CDATA[<p>最近碰到一个问题，因为一些干扰因素导致排查的时候走了一段歧路，<br>报错信息是</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">Exception in thread <span class="string">"main"</span> com.alibaba.fastjson.JSONException: scan <span class="literal">null</span> error</span><br></pre></td></tr></table></figure>
|
|
<p>用一个简单的demo来复现下</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">enum</span> <span class="title class_">DemoEnum</span> {</span><br><span class="line"></span><br><span class="line"> DEMO1(<span class="string">"demo1"</span>, <span class="string">"desc"</span>);</span><br><span class="line"> <span class="keyword">private</span> String code;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> String desc;</span><br><span class="line"></span><br><span class="line"> DemoEnum(String code, String desc) {</span><br><span class="line"> <span class="built_in">this</span>.code = code;</span><br><span class="line"> <span class="built_in">this</span>.desc = desc;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> DemoEnum <span class="title function_">getByCode</span><span class="params">(String code)</span> {</span><br><span class="line"> <span class="keyword">for</span> (DemoEnum demoEnum : values()) {</span><br><span class="line"> <span class="keyword">if</span> (demoEnum.code.equals(code)) {</span><br><span class="line"> <span class="keyword">return</span> demoEnum;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> String <span class="title function_">getCode</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> code;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">setCode</span><span class="params">(String code)</span> {</span><br><span class="line"> <span class="built_in">this</span>.code = code;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> String <span class="title function_">getDesc</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> desc;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">setDesc</span><span class="params">(String desc)</span> {</span><br><span class="line"> <span class="built_in">this</span>.desc = desc;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>先定义一个枚举类，然后有个demo类</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">FastJsonDemo</span> <span class="keyword">implements</span> <span class="title class_">Serializable</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">long</span> <span class="variable">serialVersionUID</span> <span class="operator">=</span> <span class="number">1131767138182111892L</span>;</span><br><span class="line"> <span class="keyword">private</span> String name;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> Map<DemoEnum, String> map = <span class="keyword">new</span> <span class="title class_">HashMap</span><>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> String <span class="title function_">getName</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> name;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">setName</span><span class="params">(String name)</span> {</span><br><span class="line"> <span class="built_in">this</span>.name = name;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> Map<DemoEnum, String> <span class="title function_">getMap</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> map;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">setMap</span><span class="params">(Map<DemoEnum, String> map)</span> {</span><br><span class="line"> <span class="built_in">this</span>.map = map;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>其中map的key是上面定义的枚举类，然后在main方法中做一下序列化和反序列化</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">demo</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> <span class="type">FastJsonDemo</span> <span class="variable">fastJsonDemo</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">FastJsonDemo</span>();</span><br><span class="line"> fastJsonDemo.setName(<span class="string">"nick"</span>);</span><br><span class="line"> Map<DemoEnum, String> map = <span class="keyword">new</span> <span class="title class_">HashMap</span><>();</span><br><span class="line"> map.put(DemoEnum.getByCode(<span class="string">"code"</span>), <span class="string">"null key value"</span>);</span><br><span class="line"> fastJsonDemo.setMap(map);</span><br><span class="line"> <span class="type">FastJsonDemo</span> <span class="variable">decodeFastJsonDemo</span> <span class="operator">=</span> JSONObject.parseObject(JSONObject.toJSONString(fastJsonDemo), FastJsonDemo.class);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这样就会出现这个异常</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">Exception in thread <span class="string">"main"</span> com.alibaba.fastjson.JSONException: scan <span class="literal">null</span> error</span><br><span class="line"> at com.alibaba.fastjson.parser.JSONLexerBase.scanNullOrNew(JSONLexerBase.java:<span class="number">4531</span>)</span><br><span class="line"> at com.alibaba.fastjson.parser.JSONLexerBase.nextToken(JSONLexerBase.java:<span class="number">154</span>)</span><br><span class="line"> at com.alibaba.fastjson.parser.JSONLexerBase.nextToken(JSONLexerBase.java:<span class="number">358</span>)</span><br><span class="line"> at com.alibaba.fastjson.parser.deserializer.MapDeserializer.parseMap(MapDeserializer.java:<span class="number">227</span>)</span><br><span class="line"> at com.alibaba.fastjson.parser.deserializer.MapDeserializer.deserialze(MapDeserializer.java:<span class="number">61</span>)</span><br><span class="line"> at com.alibaba.fastjson.parser.deserializer.MapDeserializer.deserialze(MapDeserializer.java:<span class="number">41</span>)</span><br><span class="line"> at com.alibaba.fastjson.parser.deserializer.FastjsonASMDeserializer_1_FastJsonDemo.deserialze(Unknown Source)</span><br><span class="line"> at com.alibaba.fastjson.parser.deserializer.JavaBeanDeserializer.deserialze(JavaBeanDeserializer.java:<span class="number">269</span>)</span><br><span class="line"> at com.alibaba.fastjson.parser.DefaultJSONParser.parseObject(DefaultJSONParser.java:<span class="number">671</span>)</span><br><span class="line"> at com.alibaba.fastjson.JSON.parseObject(JSON.java:<span class="number">365</span>)</span><br><span class="line"> at com.alibaba.fastjson.JSON.parseObject(JSON.java:<span class="number">269</span>)</span><br><span class="line"> at com.alibaba.fastjson.JSON.parseObject(JSON.java:<span class="number">488</span>)</span><br><span class="line"> at fastjson.demo.main(demo.java:<span class="number">23</span>)</span><br></pre></td></tr></table></figure>
|
|
<p>这个异常有两个原因，第一当然是在map中出现了null作为key的数据，第二就是这个key是复杂对象<br>在反序列化以后就会出现这个异常，因为最初出现这个异常是因为我改了另一个字符串字段，并且会反序列化成json就让我判断出现了误差<br>这边做一个记录。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>上次的其他 外行聊国足</title>
|
|
<url>/2022/03/06/%E4%B8%8A%E6%AC%A1%E7%9A%84%E5%85%B6%E4%BB%96-%E5%A4%96%E8%A1%8C%E8%81%8A%E5%9B%BD%E8%B6%B3/</url>
|
|
<content><![CDATA[<p>上次本来想在换车牌后面聊下这个话题，为啥要聊这个话题呢，也很简单，在地铁上看到一对猜测是情侣或者比较关系好的男女同学在聊，因为是这位男同学是大学学的工科，然后自己爱好设计绘画相关的，可能还以此赚了点钱，在地铁上讨论男的要不要好好努力把大学课程完成好，大致的观点是没必要，本来就不适合，这一段我就不说了，恋爱人的嘴，信你个鬼。<br>后面男的说在家里又跟他爹吵了关于男足的，估计是那次输了越南，实话说我不是个足球迷，对各方面技术相关也不熟，只是对包括这个人的解释和网上一些观点的看法，纯主观，这次地铁上这位说的大概意思是足球这个训练什么的很难的，要想赢越南也很难的，不是我们能嘴炮的；在网上看到一个赞同数很多的一个回答，说什么中国是个体育弱国，但是由于有一些乒乓球，跳水等小众项目比较厉害，让民众给误解了，首先我先来反驳下这个偷换概念的观点，第一所谓的体育弱国，跟我们觉得足球不应该这么差没半毛钱关系，因为体育弱国，我们的足球本来就不是顶尖的，也并不是去跟顶尖的球队去争，以足球为例，跟巴西，阿根廷，英国，德国，西班牙，意大利，法国这些足球强国，去比较，我相信没有一个足球迷会这么去做对比，因为我们足球历史最高排名是 1998 年的 37 名，最差是 100 名，把能数出来的强队都数完，估计都还不会到 37，所以根本没有跟强队去做对比，第二体育弱国，我们的体育投入是在逐年降低吗，我们是因战乱没法好好训练踢球？还是这帮傻逼就不争气，前面也说了我们足球世界排名最高 37，最低 100，那么前阵子我们输的越南是第几，目前我们的排名 77 名，越南 92 名，看明白了么，轮排名我们都不至于输越南，然后就是这个排名，这也是我想回应那位地铁上的兄弟，我觉得除了造核弹这种高精尖技术，绝大部分包含足球这类运动，遵循类二八原则，比如满分是 100 分，从 80 提到 90 分或者 90 分提到 100 分非常难，30 分提到 40 分，50 分提到 60 分我觉得都是可以凭后天努力达成的，基本不受天赋限制，这里可以以篮球来类比下，相对足球的确篮球没有那么火，或者行业市值没法比，但是也算是相对大众了，中国在篮球方面相对比较好一点，在 08 年奥运会冲进过八强，那也不是唯一的巅峰，但是我说这个其实是想说明两方面的事情，第一，像篮球一样，状态是有起起伏伏，排名也会变动，但是我觉得至少能维持一个相对稳定的总体排名和持平或者上升的趋势，这恰恰是我们这种所谓的“体育弱国”应该走的路线，第二就是去支持我的类二八原则的，可以看到我们的篮球这两年也很垃圾，排名跌到 29 了，那问题我觉得跟足球是一样的，就是不能脚踏实地，如斯科拉说的，中国篮球太缺少竞争，打得好不好都是这些人打，打输了还是照样拿钱，相对足球，篮球的技术我还是懂一些的，对比 08 年的中国男篮，的确像姚明跟王治郅这样的天赋型+努力型球员少了以后竞争力下降在所难免，但是去对比下基本功，传球，投篮，罚球稳定性，也完全不是一个水平的，这些就是我说的，可以通过努力训练拿 80 分的，只要拿到 80 分，甚至只要拿到 60 分，我觉得应该就还算对得起球迷了，就像 NBA 里球队也会有核心球员的更替，战绩起起伏伏，但是基本功这东西，防守积极性，我觉得不随核心球员的变化而变化，就像姚明这样的天赋，其实他应该还有一些先天缺陷，大脚趾较长等，但是他从 CBA 到 NBA，在 NBA 适应并且打成顶尖中锋，离不开刻苦训练，任何的成功都不是纯天赋的，必须要付出足够的努力。<br>说回足球，如果像前面那么洗地（体育弱国），那能给我维持住一个稳定的排名我也能接受，问题是我们的经济物质资源比 2000 年前应该有了质的变化，身体素质也越来越好，即使是体育弱国，这么继续走下坡路，半死不活的，不觉得是打了自己的脸么。足球也需要基本功，基本的体能，力量这些，看看现在这些国足运动员的体型，对比下女足，说实话，如果男足这些运动员都练得不错的体脂率，耐力等，成绩即使不好，也不会比现在更差。<br>纯主观吐槽，勿喷。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>运动</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>systemtap学习记录一</title>
|
|
<url>/2025/01/26/systemtap%E5%AD%A6%E4%B9%A0%E8%AE%B0%E5%BD%95%E4%B8%80/</url>
|
|
<content><![CDATA[<p>前两天拜读了章亦春大佬的关于Dynamic Tracing的文章，觉得对现在碰到的一些问题有了一些新的思考，为了能有所产出就先写一点简单的学习记录<br>首先这个systemtap类似于一个linux系统层面的探针工具，可以让用户去监控系统的各种活动<br>以阿里云的 ubuntu 22.04 为例，<br>首先我们需要安装 <code>systemtap-sdt-dev</code> 这个包，<br>然后因为通过c的方式比较常规，我是想研究下针对像 php 这种怎么去监控<br>我们需要一个编译参数带上<code>--enable-dtrace</code> 的php版本<br>然后在编译完后我们写一个简单的测试脚本</p>
|
|
<figure class="highlight php"><table><tr><td class="code"><pre><span class="line"><span class="meta"><?php</span></span><br><span class="line"><span class="function"><span class="keyword">function</span> <span class="title">a</span>(<span class="params"><span class="variable">$b</span>, <span class="variable">$c</span></span>) </span>{</span><br><span class="line"> <span class="keyword">return</span> <span class="variable">$b</span> + <span class="variable">$c</span>;</span><br><span class="line">}</span><br><span class="line"><span class="title function_ invoke__">sleep</span>(<span class="number">1</span>);</span><br><span class="line"><span class="keyword">echo</span> <span class="title function_ invoke__">a</span>(<span class="number">1</span>, <span class="number">2</span>);</span><br></pre></td></tr></table></figure>
|
|
<p>这其实主要是为了触发系统调用<br>然后重点就是systemtap的脚本，它有点类似于c的代码，结构主要是 “事件” - “处理器”<br>比如我监听到了php发起了一个系统调用，那这个时候我想把它记录下来，打印日志这种</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">probe syscall.*</span><br><span class="line">{</span><br><span class="line"> if (execname() == "php") {</span><br><span class="line"> printf ("%s(%d) open, call %s \n", execname(), pid(), name)</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里就是针对所有的系统调用，如果是由php发起调用，那就打印出来pid和系统调用的名字（name就是这里的系统调用名字）</p>
|
|
<h2 id="事件"><a href="#事件" class="headerlink" title="事件"></a>事件</h2><p><code>syscall.system_call</code> 只是其中一类事件<br>还有对虚拟文件系统的操作 <code>vfs.file_operation</code><br>内核的方法调用 <code>kernel.function("function")</code><br>比如 <code>probe kernel.function("*@net/socket.c") { }</code><br>还有可以是内核的一些追踪点<br><code>kernel.trace("tracepoint")</code><br>比如 <code>kernel.trace("kfree_skb")</code> 这个表示每次当内核中的network buffer被释放的时候触发<br>还有的是时间时间<br><code>timer events</code><br>比如</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">probe timer.s(4)</span><br><span class="line">{</span><br><span class="line"> printf("hello world\n")</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>每4秒打印一个 <code>hello world</code><br>还有这些</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">timer.ms(milliseconds)</span><br><span class="line">timer.us(microseconds)</span><br><span class="line">timer.ns(nanoseconds)</span><br><span class="line">timer.hz(hertz)</span><br><span class="line">timer.jiffies(jiffies)</span><br></pre></td></tr></table></figure>
|
|
<h2 id="处理器"><a href="#处理器" class="headerlink" title="处理器"></a>处理器</h2><p>最常规的一类就是打印信息了<br><code>printf ( ) Statements</code><br>这个跟c语言也很像，可以用<code>%s</code> 跟 <code>%d</code> 作为占位符来填充字符串和数字<br>然后就是一些定义好的取值方法<br><code>execname() (a string with the executable name)</code><br>就比如刚才的php，<br>线程id</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">tid()</span><br><span class="line">The ID of the current thread.</span><br></pre></td></tr></table></figure>
|
|
<p>用户id</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">uid()</span><br><span class="line">The ID of the current user.</span><br></pre></td></tr></table></figure>
|
|
<p>比如系统调用就直接用 <code>name</code><br>还有很多可以研究，第一篇学习就先到这里</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>系统</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>trace</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>介绍一下 RocketMQ</title>
|
|
<url>/2020/06/21/%E4%BB%8B%E7%BB%8D%E4%B8%80%E4%B8%8B-RocketMQ/</url>
|
|
<content><![CDATA[<p>说起消息队列一般Web后端做过一段时间开发的肯定会用过，在前司的时候用的是改良版的 NSQ，有点像 NOSQL 的简写版🙄，其实是个go 语言写的消息队列，<a href="https://github.com/nsqio/nsq">nsq</a> 看代码提交感觉最近更新的不是很勤，不过因为前司有专门的中间件团队，所以还是挺好用的，而且中间件团队的大牛也很厉害，一次都没碰到过丢消息之类的错误，然后现在公司用的是 RocketMQ，本着总还是要了解下的，并且消息队列也是服务端开发中一个很重要的中间件，因为不太有不需要用消息队列的后端团队了吧，原来对 nsq 也不是特别了解原理，就打算了解下 RocketMQ。</p>
|
|
<p>还是像我这样的小白专属，消息队列用来干啥，很多都是标准答案，用来削峰填谷的，这个完全对，只是我想结合场景说给像我这样的小白同学听，想想一个电商的下单功能，除了 AT 两家之外应该大部分都是接入的支付，那么下单支付完成后一般都是等支付回调，告诉你支付完成了（也有可能是失败了，或者超时了咱们主动去查），然后这个回调里我们自己的业务代码干点啥，首先比如是把订单状态改掉了，然后会有各类的操作，比如把优惠券核销了，把其他金钱相关的也核销了，把购物车里对应的商品给删了，还有更次要的，比如发个客服消息，让用户确认下地址的，给用户加积分的等等等等，想象下如果这些都是回调里一股脑儿做掉了，那可能你的代码健壮性跟相关服务的稳定性还有性能要达到一个非常高的水平才能让业务不出现异常，并且万一流量打起来了，这些重要的不重要的操作都会阻塞着，所以需要用一个消息队列，在接到回调后只处理极少的几个核心操作，完了就把这个消息丢进消息队列里，让各个业务方去消费这个消息，把客服消息发一下，给用户加个积分等等，这样子主要的业务流程需要处理的事情就少了，速度也加快了，这个例子呢不能严格算是削峰填谷的例子，不过也算是消息队列的比较典型的使用场景了，要说真实的削峰填谷的话其实可以这么理解，假如短时间内有 1w 个请求进来，系统能支持的 QPS 才 1000，那么正常情况下服务就挂了，或者被限流了，为了让服务正常，那么可以把这些请求先放进消息队列里，我服务端以拉的模式按我的处理能力来消费，这样就没啥问题了</p>
|
|
<p>扯了这么多来聊聊 RocketMQ 长啥样</p>
|
|
<p><img data-src="https://mystore-1255223546.cos.ap-chengdu.myqcloud.com/uPic/6073827-a998e005dd13967c.png" alt="6073827-a998e005dd13967c"></p>
|
|
<p>总共有四大部分：<strong>NameServer，Broker，Producer，Consumer。</strong></p>
|
|
<h4 id="NameServer"><a href="#NameServer" class="headerlink" title="NameServer"></a>NameServer</h4><p>NameServer是一个非常简单的Topic路由注册中心，其角色类似Dubbo中的zookeeper，支持Broker的动态注册与发现。主要包括两个功能：Broker管理，NameServer接受Broker集群的注册信息并且保存下来作为路由信息的基本数据。然后提供心跳检测机制，检查Broker是否还存活；路由信息管理，每个NameServer将保存关于Broker集群的整个路由信息和用于客户端查询的队列信息。然后Producer和Conumser通过NameServer就可以知道整个Broker集群的路由信息，从而进行消息的投递和消费。NameServer通常也是集群的方式部署，各实例间相互不进行信息通讯。Broker是向每一台NameServer注册自己的路由信息，所以每一个NameServer实例上面都保存一份完整的路由信息。当某个NameServer因某种原因下线了，Broker仍然可以向其它NameServer同步其路由信息，Producer,Consumer仍然可以动态感知Broker的路由的信息。</p>
|
|
<p>NameServer压力不会太大，正常情况主要负责维持心跳和提供Topic-Broker的关系数据。但有一点需要注意，Broker向Namesr发心跳时，会带上当前自己所负责的所有Topic信息，<strong>如果Topic个数太多，会导致一次心跳中，光Topic的数据就非常大，网络情况差的话，网络传输失败，心跳失败，导致Namesrv误认为Broker心跳失败。</strong></p>
|
|
<h4 id="Broker"><a href="#Broker" class="headerlink" title="Broker"></a>Broker</h4><p>Broker主要负责消息的存储、投递和查询以及服务高可用保证，为了实现这些功能，Broker包含了以下几个重要子模块。</p>
|
|
<ul>
|
|
<li>Remoting Module：整个Broker的实体，负责处理来自clients端的请求。</li>
|
|
<li>Client Manager：负责管理客户端(Producer/Consumer)和维护Consumer的Topic订阅信息</li>
|
|
<li>Store Service：提供方便简单的API接口处理消息存储到物理硬盘和查询功能。</li>
|
|
<li>HA Service：高可用服务，提供Master Broker 和 Slave Broker之间的数据同步功能。</li>
|
|
<li>Index Service：根据特定的Message key对投递到Broker的消息进行索引服务，以提供消息的快速查询。</li>
|
|
</ul>
|
|
<h6 id="Broker的特点"><a href="#Broker的特点" class="headerlink" title="Broker的特点"></a>Broker的特点</h6><p><strong>1.负载均衡：</strong>Broker上存Topic信息，<strong>Topic由多个队列组成，队列会平均分散在多个Broker上，</strong>而Producer的发送机制保证消息尽量平均分布到所有队列中，<strong>最终效果就是所有消息都平均落在每个Broker上。</strong></p>
|
|
<p><strong>2.动态伸缩能力（非顺序消息）</strong>：Broker的伸缩性体现在两个维度：Topic, Broker。</p>
|
|
<blockquote>
|
|
<p><strong>Topic维度：</strong>假如一个Topic的消息量特别大，但集群水位压力还是很低，就可以扩大该Topic的队列数，Topic的队列数跟发送、消费速度成正比。<br> <strong>Broker维度：</strong>如果集群水位很高了，需要扩容，直接加机器部署Broker就可以。Broker起来后想NameServer注册，Producer、Consumer通过NameServer发现新Broker，立即跟该Broker直连，收发消息。</p>
|
|
</blockquote>
|
|
<p><strong>3.高可用&高可靠</strong></p>
|
|
<blockquote>
|
|
<p><strong>高可用：</strong>集群部署时一般都为主备，备机实时从主机同步消息，如果其中一个主机宕机，备机提供消费服务，但不提供写服务。<br> <strong>高可靠：</strong>所有发往broker的消息，有同步刷盘和异步刷盘机制；同步刷盘时，消息写入物理文件才会返回成功，异步刷盘时，只有机器宕机，才会产生消息丢失，broker挂掉可能会发生，但是机器宕机崩溃是很少发生的，除非突然断电</p>
|
|
</blockquote>
|
|
<h4 id="Producer"><a href="#Producer" class="headerlink" title="Producer"></a>Producer</h4><p>Producer与NameServer集群中的其中一个节点（随机选择）建立长连接，定期从NameServer获取Topic路由信息，并向提供Topic 服务的Master建立长连接，且定时向Master发送心跳。Producer完全无状态，可集群部署。<br> <strong>RocketMQ提供三种发送方式：</strong></p>
|
|
<blockquote>
|
|
<p><strong>同步：</strong>在广泛的场景中使用可靠的同步传输，如重要的通知信息、短信通知、短信营销系统等。<br> <strong>异步：</strong>异步发送通常用于响应时间敏感的业务场景，发送出去即刻返回，利用回调做后续处理。<br> <strong>一次性：</strong>一次性发送用于需要中等可靠性的情况，如日志收集，发送出去即完成，不用等待发送结果，回调等等。</p>
|
|
</blockquote>
|
|
<h6 id="生产者端的负载均衡"><a href="#生产者端的负载均衡" class="headerlink" title="生产者端的负载均衡"></a>生产者端的负载均衡</h6><p>生产者发送时，会自动轮询当前所有可发送的broker，一条消息发送成功，下次换另外一个broker发送，以达到消息平均落到所有的broker上。</p>
|
|
<h4 id="Consumer"><a href="#Consumer" class="headerlink" title="Consumer"></a>Consumer</h4><p>Consumer与NameServer集群中的其中一个节点（随机选择）建立长连接，定期从NameServer获取Topic路由信息，并向提供Topic服务的Master、Slave建立长连接，且定时向Master、Slave发送心跳。Consumer既可以从Master订阅消息，也可以从Slave订阅消息，消费者在向Master拉取消息时，Master服务器会根据拉取偏移量与最大偏移量的距离（判断是否读老消息，产生读I/O），以及从服务器是否可读等因素建议下一次是从Master还是Slave拉取。</p>
|
|
<h6 id="消费者端的负载均衡"><a href="#消费者端的负载均衡" class="headerlink" title="消费者端的负载均衡"></a>消费者端的负载均衡</h6><p>先讨论消费者的消费模式，<strong>消费者有两种模式消费：集群消费，广播消费。</strong></p>
|
|
<blockquote>
|
|
<p><strong>广播消费：每个消费者消费Topic下的所有队列。</strong><br> <strong>集群消费：一个topic可以由同一个ID下所有消费者分担消费。</strong><br> 具体例子：假如TopicA有6个队列，某个消费者ID起了2个消费者实例，那么每个消费者负责消费3个队列。如果再增加一个消费者ID相同消费者实例，即当前共有3个消费者同时消费6个队列，那每个消费者负责2个队列的消费。</p>
|
|
</blockquote>
|
|
<p>消费者端的负载均衡，就是集群消费模式下，同一个ID的所有消费者实例平均消费该Topic的所有队列。</p>
|
|
<p><strong>消费者从用户角度来看有两种类型：</strong></p>
|
|
<blockquote>
|
|
<p><strong>PullConsumer：主动从brokers处拉取消息。</strong>Consumer消费的一种类型，应用通常主动调用Consumer的拉消息方法从Broker服务器拉消息、主动权由应用控制。一旦获取了批量消息，应用就会启动消费过程。<br><strong>PushConsumer：Consumer消费的一种类型，该模式下Broker收到数据后会主动推送给消费端，该消费模式一般实时性较高。</strong></p>
|
|
</blockquote>
|
|
<h4 id="补充一些概念"><a href="#补充一些概念" class="headerlink" title="补充一些概念"></a>补充一些概念</h4><p><strong>Topic：主题，表示一类消息的集合，每个主题包含若干条消息，每条消息只能属于一个主题，是RocketMQ进行消息订阅的基本单位。</strong>Topic与生产者和消费者都是非常松散的关系，一个topic可以有0个或者1个或者多个生产者向其发送消息，换句话说，一个生产者可以同时向不同和topic发送消息。从消费者的解度来说，一个topic可能被0个或者一个或者多个消费组订阅，类似的，一个消费组可以订阅一个或者多个主题只要这个消费组的实例保持他们的订阅一致。</p>
|
|
<p><strong>Message：消息</strong>消息系统所传输信息的物理载体，生产和消费数据的最小单位，每条消息必须属于一个主题。RocketMQ中每个消息拥有唯一的Message ID，且可以携带具有业务标识的Key。系统提供了通过Message ID和Key查询消息的功能。。</p>
|
|
<p><strong>Message Queue：消息队列，</strong>一个主题被化分为一个或者多个子主题（sub-topics），“消息队列”.</p>
|
|
<p><strong>Tag：标签，为消息设置的标志，用于同一主题下区分不同类型的消息。来自同一业务单元的消息，可以根据不同业务目的在同一主题下设置不同标签。标签能够有效地保持代码的清晰度和连贯性，并优化RocketMQ提供的查询系统。消费者可以根据Tag实现对不同子主题的不同消费逻辑，实现更好的扩展性。</strong>使用tag，同一业务模块不同目的的messages就可以用相同topic不同tag来标识。Tags有益于保持你的代码干净而条理清晰，同时促进使用RocketMQ提供的查询系统的效率。Topic：主题，是生产者发送的消息和消费者拉取的消息的归类。Topic与生产者和消费者都是非常松散的关系，一个topic可以有0个或者1个或者多个生产者向其发送消息，换句话说，一个生产者可以同时向不同和topic发送消息。从消费者的解度来说，一个topic可能被0个或者一个或者多个消费组订阅，类似的，一个消费组可以订阅一个或者多个主题只要这个消费组的实例保持他们的订阅一致。</p>
|
|
<p><strong>Message Order：当使用DefaultMQPushConsumer时，你需要确定消费消息的方式：</strong></p>
|
|
<blockquote>
|
|
<p>Orderly：顺序地消费消息即表示消费的消息顺序同生产者发送的顺序一致。<br> Concurrently：并行消费。指定此方式消费，信息消费的最大并行数量仅受限于每个消费者客户端指定的线程池。</p>
|
|
</blockquote>
|
|
<p><strong>Consumer Group：消费组，</strong>同一类Consumer的集合，这类Consumer通常消费同一类消息且消费逻辑一致。消费者组使得在消息消费方面，实现负载均衡和容错的目标变得非常容易。要注意的是，消费者组的消费者实例必须订阅完全相同的Topic。RocketMQ 支持两种消息模式：集群消费（Clustering）和广播消费（Broadcasting）。<br><strong>Producer Group:生产者组，</strong>同一类Producer的集合，这类Producer发送同一类消息且发送逻辑一致。如果发送的是事务消息且原始生产者在发送之后崩溃，则Broker服务器会联系同一生产者组的其他生产者实例以提交或回溯消费。</p>
|
|
<p>上面的这些我主要参考了 RocketMQ 的 GitHub 介绍和一些优秀网文的介绍，侵权请联系我删除。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>MQ</category>
|
|
<category>RocketMQ</category>
|
|
<category>消息队列</category>
|
|
<category>RocketMQ</category>
|
|
<category>中间件</category>
|
|
<category>RocketMQ</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>MQ</tag>
|
|
<tag>消息队列</tag>
|
|
<tag>RocketMQ</tag>
|
|
<tag>削峰填谷</tag>
|
|
<tag>中间件</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>不一劳永逸的临时docker镜像拉取解决办法</title>
|
|
<url>/2024/10/13/%E4%B8%8D%E4%B8%80%E5%8A%B3%E6%B0%B8%E9%80%B8%E7%9A%84%E4%B8%B4%E6%97%B6docker%E9%95%9C%E5%83%8F%E6%8B%89%E5%8F%96%E8%A7%A3%E5%86%B3%E5%8A%9E%E6%B3%95/</url>
|
|
<content><![CDATA[<p>docker镜像拉取目前是个大问题，前阵子出现了比较大规模的封禁，导致很多原有的方案无法使用，其中包括阿里云的私有镜像地址，导致我折腾了半天<br>本来是可以在 <code>https://cr.console.aliyun.com/cn-beijing/instances/mirrors</code> 上打开，阿里云用户都有各自的私有地址</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">https://xxxxxxxx.mirror.aliyuncs.com</span><br></pre></td></tr></table></figure>
|
|
<p>然后可以通过这个方式修改</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sudo mkdir -p /etc/docker</span><br><span class="line">sudo tee /etc/docker/daemon.json <<-'EOF'</span><br><span class="line">{</span><br><span class="line"> "registry-mirrors": ["https://xxxxxxxx.mirror.aliyuncs.com"]</span><br><span class="line">}</span><br><span class="line">EOF</span><br><span class="line">sudo systemctl daemon-reload</span><br><span class="line">sudo systemctl restart docker</span><br></pre></td></tr></table></figure>
|
|
<p>这样修改以后查看下日志</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">journalctl -xe --no-pager -u docker</span><br></pre></td></tr></table></figure>
|
|
<p>但还是拉不下来，后来发现还是拉不下来<br>应该也是之前被封的原因，<br>后面重要找到了一个<br>配置使用这个就可以了</p>
|
|
<figure class="highlight json"><table><tr><td class="code"><pre><span class="line"><span class="punctuation">{</span></span><br><span class="line"> <span class="attr">"registry-mirrors"</span><span class="punctuation">:</span> <span class="punctuation">[</span></span><br><span class="line"> <span class="string">"https://dockerproxy.cn"</span></span><br><span class="line"> <span class="punctuation">]</span></span><br><span class="line"><span class="punctuation">}</span></span><br></pre></td></tr></table></figure>
|
|
<p>或者拉取镜像的前缀加上这个地址<br><code>docker pull dockerproxy.cn/yangchuansheng/derper:latest</code><br>也可以这么拉取，说实话docker镜像现在真的很必要，据说这次是因为传了某些敏感的AI语音<br>参考下面的图<br><img data-src="https://img.nicksxs.me/blog/0q9ujz.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Docker</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Docker</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>tail命令小技巧</title>
|
|
<url>/2024/12/08/tail%E5%91%BD%E4%BB%A4%E5%B0%8F%E6%8A%80%E5%B7%A7/</url>
|
|
<content><![CDATA[<p>我们日常在服务器上查看日志的时候用的很多的就是tail命令，使用tail最基本的就是</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">tail filename.log</span><br></pre></td></tr></table></figure>
|
|
<p> 这种，可以看到日志文件的最后10行<br>我们一般会使用 </p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">tail -f filename.log</span><br></pre></td></tr></table></figure>
|
|
<p>可以用 <code>-f</code>来看到后续新生成的日志，可以用来观察是否有新增的异常日志或者是否有没有按预期打印新的日志<br>然后对于我们对输出的日志行数有具体要求的</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">tail -f -n 100 filename.log</span><br></pre></td></tr></table></figure>
|
|
<p>可以用 <code>-n</code> 参数，可以指定显示行数，比如想要显示100行<br>而有一个技巧就是之前看到别的同学用过</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">tail -100f filename.log</span><br></pre></td></tr></table></figure>
|
|
<p>可以和上面产生一样的效果<br>这样可以少输几个字符，也算是一个偷懒小技巧了，如果对于经常需要登录服务器看日志的也算是个效率小窍门<br>只是这样子对于比如我只想看新增的特定内容的日志来说，好像还不行<br>这里我们就可以使用组合技，结合 <code>tail</code> 和 <code>grep</code> 命令</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">tail -f filename.log | grep something </span><br></pre></td></tr></table></figure>
|
|
<p>这样就可以关注 <code>something</code> 的新增了 </p>
|
|
]]></content>
|
|
<categories>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>linux</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>向量数据库学习-介绍一下HNSW算法</title>
|
|
<url>/2024/06/23/%E4%BB%8B%E7%BB%8D%E4%B8%80%E4%B8%8BHNSW%E7%AE%97%E6%B3%95/</url>
|
|
<content><![CDATA[<p>结合AI写的一篇HNSW的介绍文章，算是向量数据库中比较核心的算法</p>
|
|
<h2 id="1-引言"><a href="#1-引言" class="headerlink" title="1. 引言"></a>1. 引言</h2><p>向量数据库在当今人工智能和机器学习领域扮演着越来越重要的角色。随着数据规模的不断扩大和应用场景的日益复杂,传统的数据库系统已经难以满足高维向量数据的快速检索需求。在这个背景下,近似最近邻(Approximate Nearest Neighbor, ANN)搜索问题成为了一个热点研究方向。</p>
|
|
<p>HNSW(Hierarchical Navigable Small World)算法是解决ANN问题的一种高效方法。它通过构建一个多层次的图结构,结合小世界网络的特性,实现了对高维向量数据的快速、精确检索。本文将深入浅出地介绍HNSW算法的核心思想、工作原理以及实际应用。</p>
|
|
<h2 id="2-HNSW算法的核心思想"><a href="#2-HNSW算法的核心思想" class="headerlink" title="2. HNSW算法的核心思想"></a>2. HNSW算法的核心思想</h2><p>HNSW算法的核心思想可以概括为三个关键点：分层结构、小世界图和近似搜索。</p>
|
|
<h3 id="分层结构"><a href="#分层结构" class="headerlink" title="分层结构"></a>分层结构</h3><p>HNSW采用了一种多层次的图结构。想象一下,如果我们要在一个大城市中找到一个特定的地址,我们通常会先看城市地图,然后逐步缩小范围到区、街道,最后定位到具体门牌号。HNSW的分层结构就类似于这种由粗到细的搜索过程。这个就是之前介绍跳表的目的，就是现在稀疏的结构里可以快速定位，然后再往下层进行搜索，提高了搜索效率<br><img data-src="https://img.nicksxs.me/blog/2c4njw.png"></p>
|
|
<h3 id="小世界图"><a href="#小世界图" class="headerlink" title="小世界图"></a>小世界图</h3><p>小世界网络是一种特殊的图结构,它具有较短的平均路径长度和较高的聚类系数。在HNSW中,每一层都是一个小世界图。这种结构使得在图中的任意两点之间都存在一条相对较短的路径,大大提高了搜索效率。</p>
|
|
<h3 id="近似搜索"><a href="#近似搜索" class="headerlink" title="近似搜索"></a>近似搜索</h3><p>与传统的KNN(K-Nearest Neighbors)算法不同,HNSW通过牺牲一小部分精度来换取显著的速度提升。它不保证一定能找到真正的最近邻,但可以以极高的概率找到足够接近的点,而且搜索速度要快得多。</p>
|
|
<h2 id="3-HNSW的工作原理"><a href="#3-HNSW的工作原理" class="headerlink" title="3. HNSW的工作原理"></a>3. HNSW的工作原理</h2><h3 id="数据结构-多层图的构建过程"><a href="#数据结构-多层图的构建过程" class="headerlink" title="数据结构:多层图的构建过程"></a>数据结构:多层图的构建过程</h3><ol>
|
|
<li>HNSW从底层开始构建,每个数据点都会出现在底层。</li>
|
|
<li>对于每个新插入的点,算法会随机决定它是否应该出现在上一层。这个过程一直持续到某一层,该点不再被选中。</li>
|
|
<li>在每一层,新点会与该层的其他点建立连接,形成小世界图结构。</li>
|
|
</ol>
|
|
<h3 id="搜索过程-自顶向下的贪婪搜索"><a href="#搜索过程-自顶向下的贪婪搜索" class="headerlink" title="搜索过程:自顶向下的贪婪搜索"></a>搜索过程:自顶向下的贪婪搜索</h3><ol>
|
|
<li>搜索从最顶层开始,选择一个随机起点。</li>
|
|
<li>在当前层进行贪婪搜索,不断移动到离目标更近的邻居。</li>
|
|
<li>当在当前层无法找到更近的点时,下降到下一层继续搜索。</li>
|
|
<li>重复这个过程直到达到底层,得到最终结果。<br><img data-src="https://img.nicksxs.me/blog/xg85mb.png"></li>
|
|
</ol>
|
|
<h3 id="插入新向量的过程"><a href="#插入新向量的过程" class="headerlink" title="插入新向量的过程"></a>插入新向量的过程</h3><ol>
|
|
<li>确定新向量应该出现在哪些层。</li>
|
|
<li>从顶层开始,在每一层执行类似于搜索的过程,找到适合连接的邻居点。</li>
|
|
<li>建立连接,同时可能需要删除一些现有连接以维持图的结构特性。<br><img data-src="https://img.nicksxs.me/blog/ytU7Ct.png"><br>论文中的算法是这样的<br><img data-src="https://img.nicksxs.me/blog/gVKHW0.png"></li>
|
|
</ol>
|
|
<ul>
|
|
<li><p>红色节点表示新插入的数据点。在这个例子中,新节点被插入到了层级0和层级1。</p>
|
|
</li>
|
|
<li><p>红色实线表示新建立的同层连接。新节点与其邻近节点建立了连接。</p>
|
|
</li>
|
|
<li><p>红色虚线表示新建立的跨层连接。新节点在不同层级之间建立了连接。</p>
|
|
</li>
|
|
<li><p>绿色虚线箭头表示插入过程的路径。这条路径展示了算法如何从顶层开始,逐层下降,最终确定新节点的插入位置。</p>
|
|
</li>
|
|
<li><p>其他颜色的节点和灰色的连接线表示原有的HNSW结构。</p>
|
|
</li>
|
|
<li><p>插入过程从顶层开始,沿着绿色虚线所示的路径向下搜索。</p>
|
|
</li>
|
|
<li><p>在每一层,算法都会寻找最近的邻居节点,并决定是否在该层插入新节点。</p>
|
|
</li>
|
|
<li><p>在本例中,新节点被插入到了层级0(基础层)和层级1。这是因为HNSW使用概率方法来决定节点应该出现在哪些层级。</p>
|
|
</li>
|
|
<li><p>插入后,新节点与其邻近节点建立连接(红色实线),以维持图的小世界特性。</p>
|
|
</li>
|
|
<li><p>同时,新节点也与其在不同层级的对应节点建立跨层连接(红色虚线),以确保层级之间的快速访问。</p>
|
|
</li>
|
|
</ul>
|
|
<h2 id="4-HNSW的性能分析"><a href="#4-HNSW的性能分析" class="headerlink" title="4. HNSW的性能分析"></a>4. HNSW的性能分析</h2><h3 id="时间复杂度分析"><a href="#时间复杂度分析" class="headerlink" title="时间复杂度分析"></a>时间复杂度分析</h3><ul>
|
|
<li>搜索时间复杂度: O(log N),其中N是数据点的总数。</li>
|
|
<li>插入时间复杂度: 平均情况下为O(log N)。</li>
|
|
</ul>
|
|
<h3 id="空间复杂度分析"><a href="#空间复杂度分析" class="headerlink" title="空间复杂度分析"></a>空间复杂度分析</h3><ul>
|
|
<li>空间复杂度: O(N log N),因为上层节点数量呈指数衰减。</li>
|
|
</ul>
|
|
<h3 id="与其他ANN算法的比较"><a href="#与其他ANN算法的比较" class="headerlink" title="与其他ANN算法的比较"></a>与其他ANN算法的比较</h3><p>相比LSH(Locality-Sensitive Hashing)和Annoy等算法,HNSW在大多数场景下都能提供更好的查询性能和更高的准确率。然而,HNSW的内存消耗相对较高,这是它的一个潜在缺点。</p>
|
|
<h2 id="5-HNSW的实际应用"><a href="#5-HNSW的实际应用" class="headerlink" title="5. HNSW的实际应用"></a>5. HNSW的实际应用</h2><h3 id="在推荐系统中的应用"><a href="#在推荐系统中的应用" class="headerlink" title="在推荐系统中的应用"></a>在推荐系统中的应用</h3><p>HNSW可以用于快速检索相似用户或物品,提高推荐系统的响应速度和准确性。例如,在音乐推荐中,可以用HNSW快速找到与用户喜好相似的歌曲。</p>
|
|
<h3 id="在图像检索中的应用"><a href="#在图像检索中的应用" class="headerlink" title="在图像检索中的应用"></a>在图像检索中的应用</h3><p>在大规模图像数据库中,HNSW可以实现快速的相似图像搜索。这在图像搜索引擎、视觉商品搜索等场景中非常有用。</p>
|
|
<h3 id="在自然语言处理中的应用"><a href="#在自然语言处理中的应用" class="headerlink" title="在自然语言处理中的应用"></a>在自然语言处理中的应用</h3><p>HNSW可以用于词嵌入或句子嵌入的快速检索,支持语义相似度计算、文本分类等任务。</p>
|
|
<h2 id="6-HNSW的优化和变种"><a href="#6-HNSW的优化和变种" class="headerlink" title="6. HNSW的优化和变种"></a>6. HNSW的优化和变种</h2><h3 id="参数调优技巧"><a href="#参数调优技巧" class="headerlink" title="参数调优技巧"></a>参数调优技巧</h3><ul>
|
|
<li>层数选择:影响搜索速度和内存使用</li>
|
|
<li>每层连接数:影响搜索准确度和构建时间</li>
|
|
<li>候选集大小:影响搜索质量和速度的平衡</li>
|
|
</ul>
|
|
<h3 id="常见的HNSW变种算法介绍"><a href="#常见的HNSW变种算法介绍" class="headerlink" title="常见的HNSW变种算法介绍"></a>常见的HNSW变种算法介绍</h3><ul>
|
|
<li>NSW-G:改进了图的构建方法,提高了搜索效率</li>
|
|
<li>HCNNG:结合了层次聚类,提高了在某些数据集上的性能</li>
|
|
</ul>
|
|
<h2 id="7-实现HNSW的挑战与解决方案"><a href="#7-实现HNSW的挑战与解决方案" class="headerlink" title="7. 实现HNSW的挑战与解决方案"></a>7. 实现HNSW的挑战与解决方案</h2><h3 id="大规模数据集的处理"><a href="#大规模数据集的处理" class="headerlink" title="大规模数据集的处理"></a>大规模数据集的处理</h3><ul>
|
|
<li>使用外部存储和分批处理</li>
|
|
<li>采用压缩技术减少内存占用</li>
|
|
</ul>
|
|
<h3 id="并行化和分布式实现"><a href="#并行化和分布式实现" class="headerlink" title="并行化和分布式实现"></a>并行化和分布式实现</h3><ul>
|
|
<li>搜索过程的并行化</li>
|
|
<li>分布式索引构建和查询</li>
|
|
</ul>
|
|
<h3 id="动态更新和删除操作的处理"><a href="#动态更新和删除操作的处理" class="headerlink" title="动态更新和删除操作的处理"></a>动态更新和删除操作的处理</h3><ul>
|
|
<li>软删除技术</li>
|
|
<li>定期重建索引</li>
|
|
</ul>
|
|
<h2 id="8-总结与展望"><a href="#8-总结与展望" class="headerlink" title="8. 总结与展望"></a>8. 总结与展望</h2><p>HNSW算法通过其独特的分层小世界图结构,在保证高准确率的同时实现了对高维向量数据的快速检索。它在推荐系统、图像检索、自然语言处理等多个领域都有广泛应用。</p>
|
|
<p>未来,向量数据库技术可能会朝着以下方向发展:</p>
|
|
<ol>
|
|
<li>更高效的压缩和索引技术</li>
|
|
<li>更好的动态更新支持</li>
|
|
<li>与深度学习模型的更紧密集成</li>
|
|
<li>针对特定领域的优化变体</li>
|
|
</ol>
|
|
<p>随着人工智能和大数据技术的不断发展,HNSW等高效的向量检索算法将在更广阔的应用场景中发挥重要作用。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>算法</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>算法</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>一个Java类型的小问题记录</title>
|
|
<url>/2024/09/01/%E4%B8%80%E4%B8%AAJava%E7%B1%BB%E5%9E%8B%E7%9A%84%E5%B0%8F%E9%97%AE%E9%A2%98%E8%AE%B0%E5%BD%95/</url>
|
|
<content><![CDATA[<p>最近发现一个小问题，感觉挺有意思，我们来看下代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> <span class="type">Integer</span> <span class="variable">oneHundredYear</span> <span class="operator">=</span> <span class="number">36500</span>;</span><br><span class="line"> System.out.println(oneHundredYear * <span class="number">24</span> * <span class="number">60</span> * <span class="number">60</span> * <span class="number">1000</span>);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>定义了个变量，用来表示100年的时间，然后计算出对应的时间戳<br>首先看下这个值</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="number">1094004736</span></span><br></pre></td></tr></table></figure>
|
|
<p>如果对时间戳这些没概念的话，可能没发现有什么问题，但是好像也不对，乘以1000了，但是尾数是736<br>这个暂时我们先不管</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> <span class="type">Integer</span> <span class="variable">oneHundredYear</span> <span class="operator">=</span> <span class="number">36500</span>;</span><br><span class="line"> System.out.println(oneHundredYear * <span class="number">24</span> * <span class="number">60</span> * <span class="number">60</span> * <span class="number">1000</span>);</span><br><span class="line"> System.out.println(System.currentTimeMillis() + oneHundredYear * <span class="number">24</span> * <span class="number">60</span> * <span class="number">60</span> * <span class="number">1000</span>);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后来看下</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="number">1094004736</span></span><br><span class="line"><span class="number">1726195843465</span></span><br></pre></td></tr></table></figure>
|
|
<p>然后我们去看下这个时间戳，按正常理解应该是2124年的8月31日<br><img data-src="https://img.nicksxs.me/blog/Ft28qc.png"><br>但实际是不对的，只是12天后，跟100年这个差距天差地别了，这是为啥呢，可能大佬一眼就看出来了这是类型转换问题，但是比如说我这么改</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> <span class="type">Integer</span> <span class="variable">oneHundredYear</span> <span class="operator">=</span> <span class="number">36500</span>;</span><br><span class="line"> System.out.println(oneHundredYear * <span class="number">24</span> * <span class="number">60</span> * <span class="number">60</span> * <span class="number">1000L</span>);</span><br><span class="line"> System.out.println(System.currentTimeMillis() + oneHundredYear * <span class="number">24</span> * <span class="number">60</span> * <span class="number">60</span> * <span class="number">1000L</span>);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>或者</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> <span class="type">Integer</span> <span class="variable">oneHundredYear</span> <span class="operator">=</span> <span class="number">36500</span>;</span><br><span class="line"> System.out.println(Long.valueOf(oneHundredYear * <span class="number">24</span> * <span class="number">60</span> * <span class="number">60</span> * <span class="number">1000</span>));</span><br><span class="line"> System.out.println(System.currentTimeMillis() + Long.valueOf(oneHundredYear * <span class="number">24</span> * <span class="number">60</span> * <span class="number">60</span> * <span class="number">1000</span>));</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>会发现还是一样，依旧是错误的<br>简单来说，就在于类型转换的时机，我们看下这个示例就知道了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> <span class="type">Integer</span> <span class="variable">oneHundredYear</span> <span class="operator">=</span> <span class="number">36500</span>;</span><br><span class="line"> System.out.println(oneHundredYear * <span class="number">24</span>);</span><br><span class="line"> System.out.println(oneHundredYear * <span class="number">24</span> * <span class="number">60</span>);</span><br><span class="line"> System.out.println(oneHundredYear * <span class="number">24</span> * <span class="number">60</span> * <span class="number">60</span>);</span><br><span class="line"> System.out.println(oneHundredYear * <span class="number">24</span> * <span class="number">60</span> * <span class="number">60</span> * <span class="number">1000</span>);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>结果是这样</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="number">876000</span></span><br><span class="line"><span class="number">52560000</span></span><br><span class="line">-<span class="number">1141367296</span></span><br><span class="line"><span class="number">1094004736</span></span><br></pre></td></tr></table></figure>
|
|
<p>到乘第二个60的时候已经是负的了，因为已经超过了Integer的范围了<br>但是为啥用后面的示例转换成long类型还不行呢<br>这个就在于编译器是怎么做类型转换的<br>在第一个oneHundredYear跟24相乘的时候是认为两个Integer相乘，并且没有检查范围<br>只有在乘以显式申明的最后的1000的时候才会做转换<br>我们看下反编译<br><img data-src="https://img.nicksxs.me/blog/17RnBg.png"><br>可以看到是先做int型的乘法，碰到有参数是long型时才会转类型<br>那么理论上其实我们只要在第二个60及之前申明long或者强转long就行了，这也是个很基础的问题，只是有时候写代码的时候直觉会以为加了个L就可以了，但实际是没那么简单</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>介绍下最近比较实用的端口转发</title>
|
|
<url>/2021/11/14/%E4%BB%8B%E7%BB%8D%E4%B8%8B%E6%9C%80%E8%BF%91%E6%AF%94%E8%BE%83%E5%AE%9E%E7%94%A8%E7%9A%84%E7%AB%AF%E5%8F%A3%E8%BD%AC%E5%8F%91/</url>
|
|
<content><![CDATA[<h3 id="vscode-扩展转发"><a href="#vscode-扩展转发" class="headerlink" title="vscode 扩展转发"></a>vscode 扩展转发</h3><p>在日常使用云服务器的时候，如果要访问上面自建的 mysql，一般要不直接开对应的端口，然后需要对本地 ip 进行授权，但是这个方案会有比较多的限制，比如本地 ip 变了，比如是非固定出口 ip 的家用宽带，或者要在家里跟公司都要访问，如果对所有 ip 都授权的话会不安全，这个时候其实是用 ssh 端口转发是个比较安全方便的方式。<br>原来在这个之前其实对这块内容不太了解，后面是听朋友说的，vscode 的 <a href="https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh"><code>Remote - SSH</code></a> 扩展可以很方便的使用端口转发，在使用该扩展的时候，会在控制台位置里都出现一个”端口” tab<br><img data-src="https://img.nicksxs.me/uPic/k51ca1.png"><br>如图中所示，我就是将一个服务器上的 mysql 的 3306 端口转发到本地的 3307 端口，至于为什么不用 3306 是因为本地我也有个 mysql 已经使用了 3306 端口，这个方法是使用的 vscode 的这个扩展，</p>
|
|
<h3 id="ssh-命令转发"><a href="#ssh-命令转发" class="headerlink" title="ssh 命令转发"></a>ssh 命令转发</h3><p>还有个方式是直接使用 ssh 命令<br>命令可以如此</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">ssh -CfNg -L 3307:127.0.0.1:3306 user1@199.199.199.199</span><br></pre></td></tr></table></figure>
|
|
<p>简单介绍下这个命令<br><code>-C</code> 表示的是压缩数据包<br><code>-f</code> 表示后台执行命令<br><code>-N</code> 是表示不执行具体命令只用于端口转发<br><code>-g</code> 表示允许远程主机连接本地转发端口<br><code>-L</code> 则是具体端口转发的映射配置<br>上面的命令就是将远程主机的 127.0.0.1:3306 对应转发到本地 3307<br>而后面的用户则就是登录主机的用户名<code>user1</code>和ip地址<code>199.199.199.199</code>，当然这个配置也不是唯一的</p>
|
|
<h3 id="ssh-config-配置转发"><a href="#ssh-config-配置转发" class="headerlink" title="ssh config 配置转发"></a>ssh config 配置转发</h3><p>还可以在ssh 的 config 配置中加对应的配置</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">Host host1</span><br><span class="line"> HostName 199.199.199.199</span><br><span class="line"> User user1</span><br><span class="line"> IdentityFile /Users/user1/.ssh/id_rsa</span><br><span class="line"> ServerAliveInterval 60</span><br><span class="line"> LocalForward 3310 127.0.0.1:3306</span><br></pre></td></tr></table></figure>
|
|
<p>然后通过 ssh host1 连接服务器的时候就能顺带做端口转发</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>ssh</category>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>ssh</tag>
|
|
<tag>端口转发</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>介绍下laravel herd工具</title>
|
|
<url>/2025/07/13/%E4%BB%8B%E7%BB%8D%E4%B8%8Blaravel-herd%E5%B7%A5%E5%85%B7/</url>
|
|
<content><![CDATA[<p>herd是在使用过了valet之后，又一款laravel出品的好用的工具，之前使用valet可以把laravel应用的本地域名解析和https加密等都处理好，是个非常不错的工具。<br>但是在我的使用场景里还有一个比较头疼的问题就是多版本php的切换，虽然php在大型网站，非常复杂的业务系统使用会有一些不足，弱类型会导致复杂逻辑的后续维护重构变得比较困难<br>只是对于个人使用来说，仍然是个非常不错的工具语言<br>同时laravel也是个很不错（除了有些过度封装）的框架，但是因为之前我自己写的工具大部分是基于php7.4的，laravel是比较古老的laravel 5.6版本，目前很多新的包都要求php版本是8.x，laravel也要10+了，一方面是我原先依赖的工具包没有对应的更新，另一方面是php7.4切换其他版本会有icu4c依赖版本的问题，挺麻烦<br>这次研究了下herd，它是可以做php版本切换的，<br><img data-src="https://img.nicksxs.me/herd_php.png"><br>像这边我装了7.4和8.2版本，并且最难的是它吧依赖也一并打包了，所以就不用管icu4c这种依赖版本的问题，省心省力<br>还可以针对site来设置不同的php版本，这个是非常方便的<br>只是它也不是完全没有问题的，在使用的时候发现一个比较奇葩的问题，就是默认运行时会有这个报错<br><code>preg_match(): Allocation of JIT memory failed</code><br>而它需要的解决方式在herd环境中变得无效，因为这里给出的错误提示是要把php的配置中的 <code>pcre.jit</code> 设置成 <code>0</code><br>我们可以通过在版本号上右键，看到对应版本php的ini配置文件所在位置<br><img data-src="https://img.nicksxs.me/herd_php_ini.png"><br>但是打开后发现这个设置已经存在<br>后来搜了一圈才发现了一个很奇葩的环境变量<br>比如是php7.4就需要设置这个<br><code>HERD_PHP_74_INI_SCAN_DIR="/Users/dasun/Library/Application Support/Herd/config/php/74/"</code><br>来源是在这个<a href="https://github.com/beyondcode/herd-community/issues/136">issue</a>里,还是个挺麻烦的问题<br>另外就是herd其实是卖收费版本的，相对来说存在一些限制，比如php的extension是不支持便捷的自定义安装的，如果有需要可能需要一些奇技淫巧，参考<a href="https://atlas.dev/blog/add-extensions-to-laravel-herd-without-homebrew">这里</a>，但是至少还是帮我解决了问题了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>php</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>php</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>介绍一批比较不错的n8n模板</title>
|
|
<url>/2025/08/10/%E4%BB%8B%E7%BB%8D%E4%B8%80%E6%89%B9%E6%AF%94%E8%BE%83%E4%B8%8D%E9%94%99%E7%9A%84n8n%E6%A8%A1%E6%9D%BF/</url>
|
|
<content><![CDATA[<p>上次介绍了n8n的安装和简单的工作流搭建，现在我们可以来看下一批免费开源的模板，也许它不能直接为我们所用，但是可以给我们搭建使用过程中提供一些帮助<br>就是这个仓库 <code>https://github.com/wassupjay/n8n-free-templates</code><br>比如我们看下第一个目录，是关于机器学习的<br>我们可以直接直接把原始文件json格式的下载下来，<br><img data-src="https://img.nicksxs.me/n8n_import.png"><br>然后就可以按自己的需求进行调整，当然还是需要一些openai的api这些才能完全体跑起来，我本地受限制就暂时不完整演示了<br>而如果觉得这些模板可能离实际使用有点远的话，可以看下官方的模板库，发现官方的模板库是更适合我们初学者的<br>官方的可以在这里找到<br><code>https://n8n.io/workflows/</code><br>上半部分是如何学习n8n<br><img data-src="https://img.nicksxs.me/n8n_template_to_learn.png"><br>下半部分是热门的template<br><img data-src="https://img.nicksxs.me/n8n_trending_template.png"><br>比如我们来看下Chat with local LLMs using n8n and Ollama <a href="https://n8n.io/workflows/2384-chat-with-local-llms-using-n8n-and-ollama/">这个</a><br>我们可以直接导入<br>然后配置ollama的端口和模型<br><img data-src="https://img.nicksxs.me/n8n_local_ollama.png"><br>这样我们就可以借助ollama来做个类似chatgpt的网页工具，当然这些只是举例，真正使用的还有待各自发掘和分享</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>从丁仲礼被美国制裁聊点啥</title>
|
|
<url>/2020/12/20/%E4%BB%8E%E4%B8%81%E4%BB%B2%E7%A4%BC%E8%A2%AB%E7%BE%8E%E5%9B%BD%E5%88%B6%E8%A3%81%E8%81%8A%E7%82%B9%E5%95%A5/</url>
|
|
<content><![CDATA[<p>几年前看了柴静的《穹顶之下》觉得这个记者调查得很深入，挺有水平，然后再看到了她跟丁仲礼的采访，其实没看完整，也没试着去理解，就觉得环境问题挺严重的，为啥柴静这个对面的这位好像对这个很不屑的样子，最近因为丁仲礼上了美国制裁名单，B 站又有人把这个视频发了出来，就完整看了下，就觉得自己挺惭愧的，就抱着对柴静的好感而没来由的否定了丁老的看法和说法，所以人也需要不断地学习，改正之前错误的观点，当然不是说我现在说的就是百分百正确，只是个人的一些浅显的见解</p>
|
|
<p>先聊聊这个事情，整体看下来我的一些理解，IPCC给中国的方案其实是个很大的陷阱，它里面有几个隐藏的点是容易被我们外行忽略的，第一点是基数，首先发达国家目前（指2010年采访或者IPCC方案时间）的人均碳排放量已经是远高于发展中国家的了，这也就导致了所谓的发达国家承诺减排80%是个非常有诚意的承诺其实就是忽悠；第二点是碳排放是个累计过程，从1900年开始到2050年，或者说到2010年，发达国家已经排的量是远超过发展中国家的，这是非常不公平的；第三点其实是通过前两点推导出来的，也就是即使发达国家这么有诚意地说减排80%，扒开这层虚伪的外衣，其实是给他们11亿人分走了48%的碳排放量，留给发展中国家55亿人口的只剩下了52%；第四点，人是否因为国家的发达与否而应受到不平等待遇，如果按国家维度，丁老说的，摩纳哥要跟中国分同样的排放量么，中国人还算不算人；第五点，这点算是我自己想的，也可能是柴静屁股决定脑袋想不到的点，她作为一个物质生活条件已经足够好了，那么对于那些生活在物质条件平均线以下的，他们是否能像城里人那样有空调地暖，洗澡有热水器浴霸，上下班能开车，这些其实都直接或者间接地导致了碳排放；他们有没有改善物质生活条件地权利呢，并且再说回来，其实丁老也给了我们觉得合理地方案，我们保证不管发达国家不管减排多少，我们都不会超过他们的80%，我觉得这是真正的诚意，而不是接着减排80%的噱头来忽悠人，也是像丁老这样的专家才能看破这个陷阱，碳排放权其实就是发展权，就是人权，中国人就不是人了么，或者说站在贫困线以下的人民是否有改善物质条件的权利，而不是说像柴静这样，只是管她自己，可能觉得小孩因为空气污染导致身体不好，所以做了穹顶之下这个纪录片，想去改善这个事情，空气污染不是说对的，只是每个国家都有这个过程，如果不发展，哪里有资源去让人活得好，活得好了是前提，然后再去各方面都改善。</p>
|
|
<p>对于这个问题其实更想说的是人的认知偏差，之前总觉得美帝是更自由民主，公平啥的，或者说觉得美帝啥都好，有种无脑愤青的感觉，外国的月亮比较圆，但是经历了像川普当选美国总统以来的各种魔幻操作，还有对于疫情的种种不可思议的美国民众的反应，其实更让人明白第一是外国的月亮没比较圆，第二是事情总是没那么简单粗暴非黑即白，美国不像原先设想地那么领先优秀，但是的确有很多方面是全球领先的，天朝也有体制所带来的优势，不可妄自菲薄，也不能忙不自大，还是要多学习知识，提升认知水平。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>吐槽</category>
|
|
<category>疫情</category>
|
|
<category>美国</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>吐槽</tag>
|
|
<tag>疫情</tag>
|
|
<tag>美国</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>从清华美院学姐聊聊我们身边的恶人</title>
|
|
<url>/2020/11/29/%E4%BB%8E%E6%B8%85%E5%8D%8E%E7%BE%8E%E9%99%A2%E5%AD%A6%E5%A7%90%E8%81%8A%E8%81%8A%E6%88%91%E4%BB%AC%E8%BA%AB%E8%BE%B9%E7%9A%84%E6%81%B6%E4%BA%BA/</url>
|
|
<content><![CDATA[<p>前几天清华美院学姐的热点火了，然后仔细看了下，其实是个学姐诬陷以为其貌不扬的男同学摸她屁股<br><img data-src="https://img.nicksxs.me/uPic/ZovTIK.jpg"><br>然后还在朋友圈发文想让他社死，我也是挺晚才知道这个词什么意思，然后后面我看到了这个图片，挺有意思的<br><img data-src="https://img.nicksxs.me/uPic/%E6%91%B8%E5%B1%81%E8%82%A1.png"><br>本来其实也没什么想聊这个的，是在 B 站看了个吐槽这个的，然后刚好晚上乘公交的时候又碰到了有点类似的问题<br>故事描述下，我们从始发站做了公交，这辆公交司机上次碰到过一回，就是会比较关注乘客的佩戴情况，主要考虑到目前国内疫情，然后这次在差不多人都坐满的情况下，可能在提示了三次让车内乘客戴好口罩，但是他指的那个中年女性还是没有反应，司机就转头比较大声指着这个乘客（中年女性）让戴好口罩，然后这个乘客（中年女性）就大声的说“我口罩是滑下来了，你指着我干嘛，你态度这么差，要吃了我一样，我要投诉你”等等，然后可能跟她一块的一个中年女性也是这么帮腔指责司机，比较基本的理解，车子里这么多乘客，假如是处于这位乘客口罩滑下来了而不自知的情况下，司机在提示了三次以后回头指着她说，我想的是没什么问题的，但是这位却反而指责这位司机指着她，并且说是态度差，要吃了她，完全是不可理喻的，并且一直喋喋不休说她口罩滑掉了有什么错，要投诉这个司机，让他可以提前退休了，在其他乘客的劝说下司机准备继续开车时，又口吐芬芳“你个傻<em>，你来打我呀”，真的是让我再次体会到了所谓的恶人先告状的又一完美呈现，后面还有个乘客还是表示要打死司机这个傻</em>，让我有点不明所以，俗话说有人是得理不饶人，前提是得理，这种理亏不饶人真的是挺让人长见识的，试想下，司机在提示三次后，这位乘客还是没有把口罩戴好，如何在不指着这位乘客的情况下能准确的提示到她呢，并且觉得语气态度不好，司机要载着一车的人，因为你这一个乘客不戴好口罩而不能正常出发，有些着急应该很正常吧，可能是平时自己在家里耀武扬威的使唤别人习惯了吧，别人不敢这么大声跟她说话，其实想想这位中年女性应该年纪不会很大，还比较时髦的吧，像一些常见的中年杭州本地人可能是不会说傻*这个词的吧。<br>杭州的公交可能是在二月份疫情还比较严重的时候是要求上车出示健康码，后面比较缓和以后只要求佩戴好口罩，但是在我们小绍兴，目前还是一律要求检验健康码和佩戴口罩，对于疫情中，并且目前阶段国内也时有报出小范围的疫情的情况下，司机尽职要求佩戴好口罩其实也是为了乘客着想，另一种情况如果司机不严格要求，万一车上有个感染者，这位中年女性被传染了，如果能找到这个司机的话，是不是想“打死”这个司机，竟然让感染者上了车，反正她自己是不可能有错的，上来就是对方态度差，要投诉，自己不戴好口罩啥都没错，我就想知道如果因为自己没戴好口罩被感染了，是不是也是司机的错，毕竟没有像仆人那样点头哈腰求着她戴好口罩。<br>再说回来，整个车就她一个人没戴好口罩，并且还有个细节，其实这个乘客是上了车之后就没戴好了，本来上车的时候是戴好的，这种比较有可能是觉得上车的时候司机那看一眼就好了，如果好好戴着口罩，一点事情都没有，唉，纯粹是太气愤了，调理逻辑什么的就忽略吧</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>吐槽</category>
|
|
<category>疫情</category>
|
|
<category>口罩</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>吐槽</tag>
|
|
<tag>疫情</tag>
|
|
<tag>公交车</tag>
|
|
<tag>口罩</tag>
|
|
<tag>杀人诛心</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>介绍一下Java的BiFunction</title>
|
|
<url>/2024/06/30/%E4%BB%8B%E7%BB%8D%E4%B8%80%E4%B8%8BJava%E7%9A%84BiFunction/</url>
|
|
<content><![CDATA[<p>java的函数式接口是java8引入的很重要的特性，也让日常代码有了比较大的风格转变<br>这里介绍下BiFunction，<br>BiFunction的代码很短</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@FunctionalInterface</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">interface</span> <span class="title class_">BiFunction</span><T, U, R> {</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Applies this function to the given arguments.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> t the first function argument</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> u the second function argument</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> the function result</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> R <span class="title function_">apply</span><span class="params">(T t, U u)</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Returns a composed function that first applies this function to</span></span><br><span class="line"><span class="comment"> * its input, and then applies the {<span class="doctag">@code</span> after} function to the result.</span></span><br><span class="line"><span class="comment"> * If evaluation of either function throws an exception, it is relayed to</span></span><br><span class="line"><span class="comment"> * the caller of the composed function.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> <V> the type of output of the {<span class="doctag">@code</span> after} function, and of the</span></span><br><span class="line"><span class="comment"> * composed function</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> after the function to apply after this function is applied</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> a composed function that first applies this function and then</span></span><br><span class="line"><span class="comment"> * applies the {<span class="doctag">@code</span> after} function</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@throws</span> NullPointerException if after is null</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">default</span> <V> BiFunction<T, U, V> <span class="title function_">andThen</span><span class="params">(Function<? <span class="built_in">super</span> R, ? extends V> after)</span> {</span><br><span class="line"> Objects.requireNonNull(after);</span><br><span class="line"> <span class="keyword">return</span> (T t, U u) -> after.apply(apply(t, u));</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>默认的方法就跟普通的FunctionalInterface一样，就有一个apply方法，只是这里的特殊点有两个</p>
|
|
<h2 id="返回值作为参数"><a href="#返回值作为参数" class="headerlink" title="返回值作为参数"></a>返回值作为参数</h2><p>这个BiFunction有三个泛型参数，T，U跟R，而这个R其实是具体方法的返回值<br>比如我们简单实现一个字符串拼接</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">BiFunction<String, String, String> biDemo = (s1, s2) -> s1 + s2;</span><br><span class="line">System.out.println(biDemo.apply(<span class="string">"hello "</span>, <span class="string">"world"</span>));</span><br></pre></td></tr></table></figure>
|
|
<p>apply返回的就是参数R</p>
|
|
<h2 id="andThen"><a href="#andThen" class="headerlink" title="andThen"></a>andThen</h2><p>这个方法是可以让结果再调用传入的方法，并返回结果</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">Function<String, Integer> funcDemo = String::length;</span><br><span class="line">System.out.println(biDemo.andThen(funcDemo).apply(<span class="string">"hello "</span>, <span class="string">"world"</span>));</span><br></pre></td></tr></table></figure>
|
|
<p>定义了个函数接口，并且给了实现，就是字符串获取长度<br>然后传给biDemo，这样就可以输出拼接后的字符串长度</p>
|
|
<h2 id="组合使用"><a href="#组合使用" class="headerlink" title="组合使用"></a>组合使用</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <T1, T2, R1, R2> R2 <span class="title function_">combination</span><span class="params">(T1 t1, T2 t2, BiFunction<T1, T2, R1> biFunction, Function<R1, R2> function)</span> {</span><br><span class="line"> <span class="keyword">return</span> biFunction.andThen(function).apply(t1, t2);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这样就把前面的两个方法串起来了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
<tag>stream</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>体验Roo Code代码生成插件</title>
|
|
<url>/2025/10/26/%E4%BD%93%E9%AA%8CRoo-Code%E4%BB%A3%E7%A0%81%E7%94%9F%E6%88%90%E6%8F%92%E4%BB%B6/</url>
|
|
<content><![CDATA[<p>之前在GLM网站看到他们家有适配各种代码生成插件，就先来体验下这个Roo Code<br>这是个在vscode插件市场里可以安装的插件，<br><img data-src="https://img.nicksxs.me/uPic/DAEVDn.png"><br>安装完在这就有个图标了<br><img data-src="https://img.nicksxs.me/uPic/1F2bx4.png"><br>然后在API Provider里选择 <code>Z AI</code><br>Z AI Entrypoint 选择 <code>China Coding Plan</code><br>Z AI API KEY 就是他们家的API KEY<br>然后选择Model，可以选择最新的glm-4.6<br>这样就配置好了<br><img data-src="https://img.nicksxs.me/uPic/Gynn2b.png"><br>还是以相同任务来做下测验<br>他会先逐个求问，来帮忙细化我前面比较粗略的prompt<br>然后会生成这样的架构计划<br><img data-src="https://img.nicksxs.me/uPic/LddXnJ.png"><br>这让我觉得有点惊喜，在开发前已经在做系分和架构设计了</p>
|
|
<p>但是在N步之后，还是有出了问题<br><code>12:23:12 AM [vite] (client) Pre-transform error: [postcss] It looks like you're trying to use </code>tailwindcss<code>directly as a PostCSS plugin. The PostCSS plugin has moved to a separate package, so to continue using Tailwind CSS with PostCSS you'll need to install</code>@tailwindcss/postcss<code> and update your PostCSS configuration. Plugin: vite:css</code></p>
|
|
<p>不过它识别到了这个错误<br><img data-src="hhttps://img.nicksxs.me/uPic/AcFEHG.png"><br>但是其实这个修复还是有问题，后续又因为token超限了，这样的一个简单应用，它用了83K的token，虽然可能设想很好，实现的还是问题比较多<br>整体看下来Roo Code应该是想解决比较大型的项目的架构设计和代码实现，但是可能因为模型能力不够，也可能是在prompt的设计中出现了问题，比如这里看下来跟tailwind的版本和具体的使用方式有关系，这一点可能跟glm模型的训练有关<br>即使我用了copilot基于claude-sonnet-4.5模型调试了挺久还是没有解决问题<br>倒不是特别悲观，只是可能很多项目都有复杂的历史包袱，要在非常核心的项目（特别是历史悠久）中使用，目前来看还需要一些时间来进步</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>介绍个非常好用的工具 pakeplus</title>
|
|
<url>/2025/05/25/%E4%BB%8B%E7%BB%8D%E4%B8%AA%E9%9D%9E%E5%B8%B8%E5%A5%BD%E7%94%A8%E7%9A%84%E5%B7%A5%E5%85%B7-pakeplus/</url>
|
|
<content><![CDATA[<p>程序员中例如我这样的主要是服务端的其实要开发一款app会面临很多问题，相对便捷的可能是做一个网页版的，套用下组件啥的，网页版的要做成app的话之前比较可行的是通过 Electron ，但是也是学习成本比较大的一件事，现在要介绍的这个pakeplus是基于rust tauri开发的一个傻瓜式地把网页转换成app的开源软件<br>首先它的github地址是 <code>https://github.com/Sjj1024/PakePlus?tab=readme-ov-file</code></p>
|
|
<p>可以在 <code>https://github.com/Sjj1024/PakePlus/releases</code> 下载对应的版本<br>下载后打开长这样<br><img data-src="https://img.nicksxs.me/pake_intro.png"><br>点击➕号就能开始创建一个app<br>然后就是填入网页的地址，比如我这边想把我常用的一个crontab工具打包成一个app，<br>那就填入地址，以及相关的软件描述，还可以控制是否开启调试等<br><img data-src="https://img.nicksxs.me/pake_crontab.png"><br>保存后就可以开始点击预览<br><img data-src="https://img.nicksxs.me/pake_crontab_demo.png"><br>这样一个简单的网页就打包成了一个mac上的app了，这里可能更多的场景是比如我就写了一个网页版的小工具，想打包成一个app来方便随时打开使用<br>比如本地的json工具等等，有些是自己定制的更趁手，还是非常不错的，记得之前这个开发者是可能做的命令行版的<br>这次是更彻底的界面化的比较傻瓜式的打包工具<br>方便很多不了解开发，命令行的同学进行使用</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>工具</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>工具</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>介绍下让bean有序执行的方法</title>
|
|
<url>/2025/06/08/%E4%BB%8B%E7%BB%8D%E4%B8%8B%E8%AE%A9bean%E6%9C%89%E5%BA%8F%E6%89%A7%E8%A1%8C%E7%9A%84%E6%96%B9%E6%B3%95/</url>
|
|
<content><![CDATA[<p>我们在用一些设计模式的时候，特别是在java跟spring生态中，有很多基于接口，会有各种不同的实现，此时如果想让实现了同一个接口的一组处理器（bean）能够按顺序处理，会有比较多种方法，这里简单介绍两种实现方式，<br>我们先定义一个接口</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">interface</span> <span class="title class_">Processor</span> {</span><br><span class="line"> <span class="keyword">void</span> <span class="title function_">process</span><span class="params">()</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后有两个实现<br>第一个实现是</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">FirstProcessor</span> <span class="keyword">implements</span> <span class="title class_">Processor</span>{</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">process</span><span class="params">()</span> {</span><br><span class="line"> System.out.println(<span class="string">"this is "</span> + <span class="built_in">this</span>.getClass().getSimpleName());</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>第二个实现是</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">SecondProcessor</span> <span class="keyword">implements</span> <span class="title class_">Processor</span>{</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">process</span><span class="params">()</span> {</span><br><span class="line"> System.out.println(<span class="string">"this is "</span> + <span class="built_in">this</span>.getClass().getSimpleName());</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>主要是为了示意，所以就把类名打了出来<br>我们通过一个http接口来调用下</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@RestController</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">OrderController</span> {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Autowired</span></span><br><span class="line"> <span class="keyword">private</span> List<Processor> processors;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@RequestMapping(value = "/order", method = RequestMethod.GET)</span></span><br><span class="line"> <span class="meta">@ResponseBody</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">test</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">for</span> (Processor processor : processors) {</span><br><span class="line"> processor.process();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里通过autowired注解进行注入，此时我们可以运行起来看下执行顺序<br>看到控制台输出的确是有序的</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">this is FirstProcessor</span><br><span class="line">this is SecondProcessor</span><br></pre></td></tr></table></figure>
|
|
<p>这个并不是spring这么智能知道我们的想法，只是刚好按字母顺序F在前<br>比如我们再加两个处理器<br>第三个</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Service</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">ThirdProcessor</span> <span class="keyword">implements</span> <span class="title class_">Processor</span>{</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">process</span><span class="params">()</span> {</span><br><span class="line"> System.out.println(<span class="string">"this is "</span> + <span class="built_in">this</span>.getClass().getSimpleName());</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>和第四个</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Service</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">FourthProcessor</span> <span class="keyword">implements</span> <span class="title class_">Processor</span>{</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">process</span><span class="params">()</span> {</span><br><span class="line"> System.out.println(<span class="string">"this is "</span> + <span class="built_in">this</span>.getClass().getSimpleName());</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后我们请求下，就发现是</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">this is FirstProcessor</span><br><span class="line">this is FourthProcessor</span><br><span class="line">this is SecondProcessor</span><br><span class="line">this is ThirdProcessor</span><br></pre></td></tr></table></figure>
|
|
<p>没有按照我们想要的顺序执行<br>那么我们可以怎么做呢</p>
|
|
<h2 id="方法一-使用order注解"><a href="#方法一-使用order注解" class="headerlink" title="方法一 使用order注解"></a>方法一 使用order注解</h2><p>通过spring的<code>org.springframework.core.annotation.Order</code> 注解</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">package</span> org.springframework.core.annotation;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> java.lang.annotation.Documented;</span><br><span class="line"><span class="keyword">import</span> java.lang.annotation.ElementType;</span><br><span class="line"><span class="keyword">import</span> java.lang.annotation.Retention;</span><br><span class="line"><span class="keyword">import</span> java.lang.annotation.RetentionPolicy;</span><br><span class="line"><span class="keyword">import</span> java.lang.annotation.Target;</span><br><span class="line"></span><br><span class="line"><span class="meta">@Retention(RetentionPolicy.RUNTIME)</span></span><br><span class="line"><span class="meta">@Target({ElementType.TYPE, ElementType.METHOD, ElementType.FIELD})</span></span><br><span class="line"><span class="meta">@Documented</span></span><br><span class="line"><span class="keyword">public</span> <span class="meta">@interface</span> Order {</span><br><span class="line"> <span class="type">int</span> <span class="title function_">value</span><span class="params">()</span> <span class="keyword">default</span> Integer.MAX_VALUE;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>通过value值来指定顺序，值越小，顺序越靠前<br>比如我们这</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Service</span></span><br><span class="line"><span class="meta">@Order(value = 1)</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">FirstProcessor</span> <span class="keyword">implements</span> <span class="title class_">Processor</span>{</span><br><span class="line"><span class="meta">@Service</span></span><br><span class="line"><span class="meta">@Order(value = 2)</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">SecondProcessor</span> <span class="keyword">implements</span> <span class="title class_">Processor</span>{</span><br><span class="line"><span class="meta">@Service</span></span><br><span class="line"><span class="meta">@Order(value = 3)</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">ThirdProcessor</span> <span class="keyword">implements</span> <span class="title class_">Processor</span>{</span><br><span class="line"><span class="meta">@Service</span></span><br><span class="line"><span class="meta">@Order(value = 4)</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">FourthProcessor</span> <span class="keyword">implements</span> <span class="title class_">Processor</span>{</span><br></pre></td></tr></table></figure>
|
|
<p>运行验证下的确是按照我们想要的顺序执行了</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">this is FirstProcessor</span><br><span class="line">this is SecondProcessor</span><br><span class="line">this is ThirdProcessor</span><br><span class="line">this is FourthProcessor</span><br></pre></td></tr></table></figure>
|
|
<h2 id="方法二-实现-Ordered-接口"><a href="#方法二-实现-Ordered-接口" class="headerlink" title="方法二 实现 Ordered 接口"></a>方法二 实现 Ordered 接口</h2><p><code>org.springframework.core.Ordered</code> 接口，需要实现 getOrder 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">interface</span> <span class="title class_">Ordered</span> {</span><br><span class="line"> <span class="type">int</span> <span class="variable">HIGHEST_PRECEDENCE</span> <span class="operator">=</span> Integer.MIN_VALUE;</span><br><span class="line"> <span class="type">int</span> <span class="variable">LOWEST_PRECEDENCE</span> <span class="operator">=</span> Integer.MAX_VALUE;</span><br><span class="line"></span><br><span class="line"> <span class="type">int</span> <span class="title function_">getOrder</span><span class="params">()</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>我们就在处理器里实现这个接口方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">FirstProcessor</span> <span class="keyword">implements</span> <span class="title class_">Processor</span>, Ordered {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">process</span><span class="params">()</span> {</span><br><span class="line"> System.out.println(<span class="string">"this is "</span> + <span class="built_in">this</span>.getClass().getSimpleName());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="type">int</span> <span class="title function_">getOrder</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>运行下同样可以实现这个效果</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">this is FirstProcessor</span><br><span class="line">this is SecondProcessor</span><br><span class="line">this is ThirdProcessor</span><br><span class="line">this is FourthProcessor</span><br></pre></td></tr></table></figure>]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>体验心流助手iflow</title>
|
|
<url>/2025/11/02/%E4%BD%93%E9%AA%8C%E5%BF%83%E6%B5%81%E5%8A%A9%E6%89%8Biflow/</url>
|
|
<content><![CDATA[<p>最近一直在找这些AI编程助手，想找个cc的八分平替，刚好前阵子在朋友圈看到iflow-cli，据说是可以免费用GLM4.6这些模型，<br>首先安装也很简单<br>使用这行命令就行</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">bash -c "$(curl -fsSL https://gitee.com/iflow-ai/iflow-cli/raw/main/install.sh)"</span><br></pre></td></tr></table></figure>
|
|
<p>但是这里他会判断nodejs的版本，需要22及以上版本<br>这个可以用nvm切换，不过这里比较好奇，应该是nodejs的普及率太高了所以都用它做来会更方便<br>切换以后运行上面的命令，首次运行直接回进去类似于claude code的交互界面，<br>当然我还是让它给我修复下roo代码<br><img data-src="https://img.nicksxs.me/uPic/mF1bTB.png"><br>第一个错误是启动<code>npm run dev</code> 的时候就报的，后面是在chrome控制台的<br><img data-src="https://img.nicksxs.me/uPic/aNl65f.png"><br>这两个问题改完之后终于这个“庞大的”工程终于起来了，原因只是为了做roocode测试，并且我的prompt加了“精美的”这种关键词，结果用copilot改了半天index.css中依赖tailwind的问题，<br>这次总算用iflow-cli给修好了，但也是能跑起来<br><img data-src="https://273d100fa7947b37e52ceb4bedab7c0a.r2.cloudflarestorage.com/blog/uPic/6OqBrO.png"><br>可以看到页面是比之前的都精美了，但是样式还有点不对，有点偏右，而且经过copilot的修改，可能配色也已经不太一样了<br>iflow总结用下来有两个问题，第一个是有时候会莫名其妙超时，非常久，有一次我放着没管，一晚上都没结果<br>第二个是广告，在等待prompt结果的时候，会有一行文字的广告，这个体验非常差，当然如果是为了免费也没办法，希望能有一定的控制，后续可以看看它支持的模型里哪个生成的比较好</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>体验下Java 21的虚拟线程-协程</title>
|
|
<url>/2024/08/18/%E4%BD%93%E9%AA%8C%E4%B8%8BJava-21%E7%9A%84%E8%99%9A%E6%8B%9F%E7%BA%BF%E7%A8%8B-%E5%8D%8F%E7%A8%8B/</url>
|
|
<content><![CDATA[<p>Java在后续版本中添加了虚拟线程，也是类似于php跟go的协程，对应操作系统的线程是在线程基础上模拟了一层子线程的逻辑，因为减少了操作系统的线程上下文切换开销，能够在常规业务场景带了比较大的性能提升，但也并非银弹，不能包治百病<br>首先安装下jdk 21 版本，需要用 <code>/usr/libexec/java_home</code> 切换下JAVA_HOME<br>然后在PATH中设置好</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">export JAVA_HOME=$(/usr/libexec/java_home -v 21)</span><br></pre></td></tr></table></figure>
|
|
<p>首先是试一下线程版本</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">long</span> <span class="variable">start</span> <span class="operator">=</span> System.currentTimeMillis();</span><br><span class="line"><span class="type">ExecutorService</span> <span class="variable">executor</span> <span class="operator">=</span> Executors.newFixedThreadPool(<span class="number">200</span>);</span><br><span class="line"><span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < <span class="number">100000</span>; i++) {</span><br><span class="line"> executor.submit(() -> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 线程睡眠 10ms，可以等同于模拟业务耗时10ms</span></span><br><span class="line"> TimeUnit.MILLISECONDS.sleep(<span class="number">10</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"> });</span><br><span class="line">}</span><br><span class="line">executor.close();</span><br><span class="line">System.out.printf(<span class="string">"totalMillis：%dms\n"</span>, System.currentTimeMillis() - start);</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/blog/21n0D9.png"><br>耗时5897ms</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">long</span> <span class="variable">start</span> <span class="operator">=</span> System.currentTimeMillis();</span><br><span class="line"><span class="type">ExecutorService</span> <span class="variable">executor</span> <span class="operator">=</span> Executors.newVirtualThreadPerTaskExecutor();</span><br><span class="line"><span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < <span class="number">10000</span>; i++) {</span><br><span class="line"> executor.submit(() -> {</span><br><span class="line"> <span class="comment">// 模拟业务处理</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> TimeUnit.MILLISECONDS.sleep(<span class="number">10</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"> });</span><br><span class="line">}</span><br><span class="line">executor.close();</span><br><span class="line">System.out.printf(<span class="string">"totalMillis：%dms\n"</span>, System.currentTimeMillis() - start);</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/blog/C37wCQ.png"><br>耗时154ms<br>相对来说还是能快很多的<br>而核心的虚拟线程实现主要来自于调度</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">static</span> Thread <span class="title function_">newVirtualThread</span><span class="params">(Executor scheduler,</span></span><br><span class="line"><span class="params"> String name,</span></span><br><span class="line"><span class="params"> <span class="type">int</span> characteristics,</span></span><br><span class="line"><span class="params"> Runnable task)</span> {</span><br><span class="line"> <span class="keyword">if</span> (ContinuationSupport.isSupported()) {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">VirtualThread</span>(scheduler, name, characteristics, task);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">if</span> (scheduler != <span class="literal">null</span>)</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">UnsupportedOperationException</span>();</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">BoundVirtualThread</span>(name, characteristics, task);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>在创建虚拟线程过程中，我们需要去处理调度器，初始时调度器为空</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">VirtualThread(Executor scheduler, String name, <span class="type">int</span> characteristics, Runnable task) {</span><br><span class="line"> <span class="built_in">super</span>(name, characteristics, <span class="comment">/*bound*/</span> <span class="literal">false</span>);</span><br><span class="line"> Objects.requireNonNull(task);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// choose scheduler if not specified</span></span><br><span class="line"> <span class="keyword">if</span> (scheduler == <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">Thread</span> <span class="variable">parent</span> <span class="operator">=</span> Thread.currentThread();</span><br><span class="line"> <span class="keyword">if</span> (parent <span class="keyword">instanceof</span> VirtualThread vparent) {</span><br><span class="line"> scheduler = vparent.scheduler;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> scheduler = DEFAULT_SCHEDULER;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.scheduler = scheduler;</span><br><span class="line"> <span class="built_in">this</span>.cont = <span class="keyword">new</span> <span class="title class_">VThreadContinuation</span>(<span class="built_in">this</span>, task);</span><br><span class="line"> <span class="built_in">this</span>.runContinuation = <span class="built_in">this</span>::runContinuation;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>而这个默认调度器就是</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">ForkJoinPool</span> <span class="variable">DEFAULT_SCHEDULER</span> <span class="operator">=</span> createDefaultScheduler();</span><br></pre></td></tr></table></figure>
|
|
<p>对应创建的就是个<code>ForkJoinPool</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> ForkJoinPool <span class="title function_">createDefaultScheduler</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">ForkJoinWorkerThreadFactory</span> <span class="variable">factory</span> <span class="operator">=</span> pool -> {</span><br><span class="line"> PrivilegedAction<ForkJoinWorkerThread> pa = () -> <span class="keyword">new</span> <span class="title class_">CarrierThread</span>(pool);</span><br><span class="line"> <span class="keyword">return</span> AccessController.doPrivileged(pa);</span><br><span class="line"> };</span><br><span class="line"> PrivilegedAction<ForkJoinPool> pa = () -> {</span><br><span class="line"> <span class="type">int</span> parallelism, maxPoolSize, minRunnable;</span><br><span class="line"> <span class="type">String</span> <span class="variable">parallelismValue</span> <span class="operator">=</span> System.getProperty(<span class="string">"jdk.virtualThreadScheduler.parallelism"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">maxPoolSizeValue</span> <span class="operator">=</span> System.getProperty(<span class="string">"jdk.virtualThreadScheduler.maxPoolSize"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">minRunnableValue</span> <span class="operator">=</span> System.getProperty(<span class="string">"jdk.virtualThreadScheduler.minRunnable"</span>);</span><br><span class="line"> <span class="keyword">if</span> (parallelismValue != <span class="literal">null</span>) {</span><br><span class="line"> parallelism = Integer.parseInt(parallelismValue);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> parallelism = Runtime.getRuntime().availableProcessors();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (maxPoolSizeValue != <span class="literal">null</span>) {</span><br><span class="line"> maxPoolSize = Integer.parseInt(maxPoolSizeValue);</span><br><span class="line"> parallelism = Integer.min(parallelism, maxPoolSize);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> maxPoolSize = Integer.max(parallelism, <span class="number">256</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (minRunnableValue != <span class="literal">null</span>) {</span><br><span class="line"> minRunnable = Integer.parseInt(minRunnableValue);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> minRunnable = Integer.max(parallelism / <span class="number">2</span>, <span class="number">1</span>);</span><br><span class="line"> }</span><br><span class="line"> Thread.<span class="type">UncaughtExceptionHandler</span> <span class="variable">handler</span> <span class="operator">=</span> (t, e) -> { };</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">asyncMode</span> <span class="operator">=</span> <span class="literal">true</span>; <span class="comment">// FIFO</span></span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">ForkJoinPool</span>(parallelism, factory, handler, asyncMode,</span><br><span class="line"> <span class="number">0</span>, maxPoolSize, minRunnable, pool -> <span class="literal">true</span>, <span class="number">30</span>, SECONDS);</span><br><span class="line"> };</span><br><span class="line"> <span class="keyword">return</span> AccessController.doPrivileged(pa);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>可以看到参数都是通过系统参数获取，或者用系统的cpu数量来决定并行度，主体的逻辑就是既然系统线程开销大，那我就在系统线程内部模拟一个更小颗粒度的，在线程内部进行调度的模型，以此来减少系统切换开销，只不过细节还有很多需要研究，有兴趣的可以留言探讨</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>体验openai开源的gpt-oss模型</title>
|
|
<url>/2025/08/30/%E4%BD%93%E9%AA%8Copenai%E5%BC%80%E6%BA%90%E7%9A%84gpt-oss%E6%A8%A1%E5%9E%8B/</url>
|
|
<content><![CDATA[<p>openai终于履行诺言开源了自家模型，只不过不是像deepseek那样把最新最好的开源出来，而是专门调了一大一小两个模型来开源，当然也有一些技术亮点<br>刚好我本地是18g内存的mbp，勉强可以跑20b大小的，这次开源的有20b跟200b感觉尺寸上还是比较取巧的，20b适合个人本地使用，稍微扩下内存一般就能用了，只是速度稍慢些<br>官方介绍的引用</p>
|
|
<blockquote>
|
|
<p>Gpt-oss-120b 模型在核心推理基准测试中与 OpenAI o4-mini 模型几乎持平，同时能在单个 80GB GPU 上高效运行。Gpt-oss-20b 模型在常见基准测试中与 OpenAI o3‑mini 模型取得类似结果，且可在仅配备 16GB 内存的边缘设备上运行，使其成为设备端应用、本地推理或无需昂贵基础设施的快速迭代的理想选择。这两个模型在工具使用、少样本函数调用、CoT推理（如在 Tau-Bench 智能体评估套件中的结果所示）以及 HealthBench 测试中表现强劲（甚至超越了 OpenAI o1 和 GPT‑4o 等专有模型）。 </p>
|
|
</blockquote>
|
|
<h2 id="预训练和模型架构"><a href="#预训练和模型架构" class="headerlink" title="预训练和模型架构"></a>预训练和模型架构</h2><p>gpt-oss 模型采用我们最先进的预训练和后训练技术进行训练，特别注重推理能力、效率以及在各种部署环境中的实际应用性。虽然我们已经公开发布了包括 Whisper 和 CLIP 在内的其他模型，但 gpt-oss 模型是我们自 GPT‑2[1]以来的首个开放大型语言模型。</p>
|
|
<p>每个模型都是一个 Transformer，它利用专家混合 (MoE[2]) 来减少处理输入所需的活跃参数数量。Gpt-oss-120b 每个令牌激活 51 亿个参数，而 gpt-oss-20b 激活 36 亿个参数。这两个模型的总参数数分别为 1,170 亿和 210 亿。这些模型采用交替的密集和局部带状稀疏注意力模式，与 GPT‑3 [3] 类似。为了提高推理和内存效率，这些模型还使用分组多查询注意力，分组大小为 8。我们使用旋转位置嵌入 (RoPE[4]) 进行位置编码，并原生支持长达 128k 的上下文长度。</p>
|
|
<table>
|
|
<thead>
|
|
<tr>
|
|
<th>模型</th>
|
|
<th>层数</th>
|
|
<th>总参数</th>
|
|
<th>每个令牌的活跃参数</th>
|
|
<th>总专家数</th>
|
|
<th>每个令牌的活跃专家数</th>
|
|
<th>上下文长度</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr>
|
|
<td>gpt-oss-120b</td>
|
|
<td>36</td>
|
|
<td>117b</td>
|
|
<td>5.1b</td>
|
|
<td>128</td>
|
|
<td>4</td>
|
|
<td>128k</td>
|
|
</tr>
|
|
<tr>
|
|
<td>gpt-oss-20b</td>
|
|
<td>24</td>
|
|
<td>21b</td>
|
|
<td>3.6b</td>
|
|
<td>32</td>
|
|
<td>4</td>
|
|
<td>128k</td>
|
|
</tr>
|
|
</tbody></table>
|
|
<p>比如我本地用ollama来运行了20b大小的，</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">ollama run gpt-oss:20b</span><br></pre></td></tr></table></figure>
|
|
<p>运行起来大概是占用了13g多点的内存，因为mac本身也有不少占用内存的以及我自己的一些后台运行的软件，所以比较捉襟见肘，有条件的上32g的mac mini可能比较富裕点<br>虽然现在ollama已经带了gui界面，但是发现运行的特别慢，直接通过命令行倒还是可以接受，比如我们问个Java相关的问题</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_">></span><span class="language-bash">>> 怎么成为Java架构师</span></span><br></pre></td></tr></table></figure>
|
|
<p>首先是思维过程</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">Thinking...</span><br><span class="line">The user asks in Chinese: "怎么成为Java架构师" which means "How to become a Java architect?" They likely want steps, skills, learning path. Provide a thorough</span><br><span class="line">guide: fundamentals, design patterns, architecture frameworks, deep dive, system design, cloud, CI/CD, soft skills, leadership, etc. Also talk about</span><br><span class="line">certifications, real-world experience, project, mentorship, portfolio. Cover common architecture patterns: microservices, distributed, event-driven. Tools:</span><br><span class="line">Spring, Hibernate, etc. Should also mention reading books: Clean Architecture, Domain-Driven Design, Patterns. Also cover continuous learning, speaking at</span><br><span class="line">conferences, writing blogs.</span><br><span class="line"></span><br><span class="line">Let's structure answer: introduction, prerequisites, technical skills, architectural concepts, real-world experience, soft skills, certifications, community,</span><br><span class="line">portfolio, continuous learning. Provide resources and timelines. Since Chinese user, answer in Chinese. Provide details, resources links. Make it thorough. Use</span><br><span class="line">bullet points for clarity.</span><br><span class="line"></span><br><span class="line">Let's now produce final.</span><br><span class="line">...done thinking.</span><br></pre></td></tr></table></figure>
|
|
<p>然后就是给出结果，体感是个位数token每秒的输出速度</p>
|
|
<blockquote>
|
|
<p>以下就是生成的内容 </p>
|
|
</blockquote>
|
|
<h2 id="目标：成为一名-Java-架构师"><a href="#目标：成为一名-Java-架构师" class="headerlink" title="目标：成为一名 Java 架构师"></a>目标：成为一名 <strong>Java 架构师</strong></h2><blockquote>
|
|
<p><em>架构师不是一次获得“技术标签”，而是一种综合的 <strong>设计思维</strong> 与 <strong>业务洞察</strong> 的能力。</em></p>
|
|
</blockquote>
|
|
<p>下面给你一条完整、可量化的路线图，帮助你从“有经验的 Java 开发者”成长为 <strong>架构师</strong>。</p>
|
|
<blockquote>
|
|
<p><strong>小贴士</strong>：如果你手头已有 3–5 年的项目经验，就可以直接对应后面的“实战经验”板块。如果你刚起步，建议先走“技术打底”阶段，然后再投身实战。</p>
|
|
</blockquote>
|
|
<hr>
|
|
<h2 id="一、技术打底（1–2-年）"><a href="#一、技术打底（1–2-年）" class="headerlink" title="一、技术打底（1–2 年）"></a>一、技术打底（1–2 年）</h2><table>
|
|
<thead>
|
|
<tr>
|
|
<th>目标</th>
|
|
<th>必备技术 / 书籍 / 练手</th>
|
|
<th>练习建议</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr>
|
|
<td><strong>Java 基础</strong></td>
|
|
<td>Java SE 8+（lambda, stream, JPA 等）</td>
|
|
<td>先练手 OOP、设计模式、并发，做 Leetcode、练下 JDK 内部实现（如 <code>ArrayList</code>、<code>HashMap</code>）</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>框架</strong></td>
|
|
<td>Spring Framework（MVC、AOP）、Spring Boot、Spring Cloud、Spring Data JPA</td>
|
|
<td>搭建多层 CRUD + MVC + Security + Redis + MySQL 的完整项目</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>ORM</strong></td>
|
|
<td>Hibernate / JPA</td>
|
|
<td>写复杂的查询，掌握 Fetch 策略、缓存、懒加载</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>数据库</strong></td>
|
|
<td>关系型（MySQL/PostgreSQL） + NoSQL（MongoDB / Redis）</td>
|
|
<td>学习分库分表、事务、读写分离</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>安全</strong></td>
|
|
<td>OAuth2 / JWT / Spring Security</td>
|
|
<td>单独做一套认证系统</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>单元测试</strong></td>
|
|
<td>JUnit、Mockito</td>
|
|
<td>代码覆盖率≥90%</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>CI/CD</strong></td>
|
|
<td>Git, Maven/Gradle, Docker, Jenkins/GitHub Actions</td>
|
|
<td>自动化构建、单元/集成测试、镜像推送</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>监控</strong></td>
|
|
<td>Micrometer + Prometheus + Grafana / ELK</td>
|
|
<td>记录度量、日志、告警</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>微服务</strong></td>
|
|
<td>Spring Cloud（Nacos/Spring Cloud Config + Eureka/Consul + Zuul/Gateway + Hystrix）</td>
|
|
<td>搭建 3‑5 个小服务，练习服务间调用与降级</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>消息中间件</strong></td>
|
|
<td>Kafka / RabbitMQ</td>
|
|
<td>读写队列，练习发布/订阅，事务与回滚</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>分布式事务</strong></td>
|
|
<td>Saga、TCC、XA</td>
|
|
<td>通过 Spring Transaction、Atomikos 练习</td>
|
|
</tr>
|
|
</tbody></table>
|
|
<p><strong>学习资源（中文 / 英文）</strong></p>
|
|
<table>
|
|
<thead>
|
|
<tr>
|
|
<th>主题</th>
|
|
<th>书籍</th>
|
|
<th>课程</th>
|
|
<th>文章 / 官网</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr>
|
|
<td>Java</td>
|
|
<td>《Java 编程思想》, 《深入理解 Java 虚拟机》</td>
|
|
<td>《Java 8+ 入门与进阶》（慕课, 极客时间）</td>
|
|
<td>Oracle JDK 官网, 《Effective Java》解读</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Spring</td>
|
|
<td>《Spring 实战 5》, 《Spring Cloud 实战》</td>
|
|
<td>Spring 官方培训，Bilibili / 慕课</td>
|
|
<td>Spring 官网 & Spring Cloud 官网</td>
|
|
</tr>
|
|
<tr>
|
|
<td>微服务</td>
|
|
<td>《微服务设计》, 《Domain-Driven Design》</td>
|
|
<td>《Spring Cloud 全栈》（极客时间）</td>
|
|
<td>微服务架构图 (Zhou’s blog, DZone)</td>
|
|
</tr>
|
|
<tr>
|
|
<td>大数据 / Kafka</td>
|
|
<td>《Kafka 权威指南》</td>
|
|
<td>《Kafka 入门 & 进阶》</td>
|
|
<td>Apache Kafka 官网</td>
|
|
</tr>
|
|
<tr>
|
|
<td>系统设计</td>
|
|
<td>《系统架构设计》, 《分布式系统设计之道》</td>
|
|
<td>《系统设计基础》课程</td>
|
|
<td>阿里云、腾讯云白皮书</td>
|
|
</tr>
|
|
</tbody></table>
|
|
<blockquote>
|
|
<p><strong>练手项目</strong>：做一个 <strong>电商平台</strong> 或 <strong>社交系统</strong>，分拆为多个服务：用户服务、商品服务、订单服务、支付服务、物流服务等。通过实现 **统一鉴权、限流、熔断、日志聚合<br>**，让自己彻底体验架构层面的痛点。</p>
|
|
</blockquote>
|
|
<hr>
|
|
<h2 id="二、系统设计能力（3–5-年经验）"><a href="#二、系统设计能力（3–5-年经验）" class="headerlink" title="二、系统设计能力（3–5 年经验）"></a>二、系统设计能力（3–5 年经验）</h2><h3 id="1-业务需求分析"><a href="#1-业务需求分析" class="headerlink" title="1. 业务需求分析"></a>1. 业务需求分析</h3><ul>
|
|
<li><strong>痛点</strong>：业务需求往往是模糊、频繁变动的，架构师需要把业务拆解为 <strong>核心能力</strong> 与 <strong>边缘功能</strong>。</li>
|
|
<li><strong>实践</strong>：每月挑选一个业务点（如优惠券、库存预警等），写 1–2 页面需求文档，然后把需求拆解成 <strong>业务领域模型</strong>。</li>
|
|
</ul>
|
|
<h3 id="2-设计模式与架构模式"><a href="#2-设计模式与架构模式" class="headerlink" title="2. 设计模式与架构模式"></a>2. 设计模式与架构模式</h3><table>
|
|
<thead>
|
|
<tr>
|
|
<th>类别</th>
|
|
<th>经典模式</th>
|
|
<th>示例</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr>
|
|
<td>业务层</td>
|
|
<td>领域模型 (Domain Model), 领域事件 (Domain Event)</td>
|
|
<td>商品模块使用领域事件实现库存同步</td>
|
|
</tr>
|
|
<tr>
|
|
<td>持久层</td>
|
|
<td>DAO, Repository, Unit of Work</td>
|
|
<td>JPA Repository + UnitOfWork</td>
|
|
</tr>
|
|
<tr>
|
|
<td>基础设施</td>
|
|
<td>适配器 (Adapter), 代理 (Proxy), 桥接 (Bridge)</td>
|
|
<td>用 Feign+Ribbon 代理调用第三方支付</td>
|
|
</tr>
|
|
<tr>
|
|
<td>核心架构</td>
|
|
<td>微服务, 事件驱动, Command Query Responsibility Segregation (CQRS), Event Sourcing, Saga</td>
|
|
<td>微服务 + Kafka</td>
|
|
</tr>
|
|
<tr>
|
|
<td>运行时</td>
|
|
<td>延迟策略 (Retry), 熔断 (Circuit Breaker), 限流 (Limiter), 负载均衡 (LoadBalancer)</td>
|
|
<td>Hystrix + Ribbon + Sentinel</td>
|
|
</tr>
|
|
</tbody></table>
|
|
<blockquote>
|
|
<p><strong>书单</strong>：</p>
|
|
<ul>
|
|
<li>《Design Patterns》 经典翻译版</li>
|
|
<li>《架构师的选型与设计》</li>
|
|
<li>《系统架构师的自我修养》</li>
|
|
<li>《大规模分布式系统架构》（阿里巴巴架构师经验分享）</li>
|
|
</ul>
|
|
</blockquote>
|
|
<h3 id="3-云原生技术"><a href="#3-云原生技术" class="headerlink" title="3. 云原生技术"></a>3. 云原生技术</h3><ul>
|
|
<li><strong>容器</strong>: Docker、Kubernetes；自研或使用 <strong>OpenJDK + JDK 21</strong>。</li>
|
|
<li><strong>服务网格</strong>：Istio/Linkerd。</li>
|
|
<li><strong>CI/CD</strong>：ArgoCD + Jenkins。</li>
|
|
<li><strong>自动化</strong>：Helm、Terraform、Ansible。</li>
|
|
<li><strong>云服务</strong>：Aliyun、AWS、Azure，熟悉至少一个云的部署与运维。</li>
|
|
</ul>
|
|
<h3 id="4-监控、日志、运维"><a href="#4-监控、日志、运维" class="headerlink" title="4. 监控、日志、运维"></a>4. 监控、日志、运维</h3><ul>
|
|
<li><strong>分布式追踪</strong>：OpenTelemetry + Zipkin/Jaeger。</li>
|
|
<li><strong>AIOps</strong>：使用 <strong>Prometheus + Grafana</strong>，结合 Alertmanager。</li>
|
|
<li><strong>日志</strong>：ELK、Loki，或 <strong>OpenTelemetry Log</strong>。</li>
|
|
</ul>
|
|
<h3 id="5-软技能-团队协作"><a href="#5-软技能-团队协作" class="headerlink" title="5. 软技能 & 团队协作"></a>5. 软技能 & 团队协作</h3><table>
|
|
<thead>
|
|
<tr>
|
|
<th>领域</th>
|
|
<th>关键能力</th>
|
|
<th>学习路径</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr>
|
|
<td><strong>产品思维</strong></td>
|
|
<td>理解业务目标、价值主张</td>
|
|
<td>与 PO、BA 讨论，用业务故事拆解</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>沟通</strong></td>
|
|
<td>需求梳理、技术方案说明</td>
|
|
<td>每周至少 1 次技术分享或午餐会</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>领导</strong></td>
|
|
<td>组内技术指导、冲刺规划</td>
|
|
<td>作为 scrum master 或技术负责人</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>安全 & 合规</strong></td>
|
|
<td>数据加密、GDPR/HIPAA</td>
|
|
<td>学习安全框架、法规手册</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>成本管理</strong></td>
|
|
<td>云成本、容量规划</td>
|
|
<td>通过 Cloud Cost Estimator 评估</td>
|
|
</tr>
|
|
</tbody></table>
|
|
<blockquote>
|
|
<p><strong>练习</strong>：把上面 4 大主题写成 PPT，给同事做一个半小时的技术分享，随后收集反思与改进。</p>
|
|
</blockquote>
|
|
<hr>
|
|
<h2 id="三、职业路线（5-10-岁经验）"><a href="#三、职业路线（5-10-岁经验）" class="headerlink" title="三、职业路线（5-10 岁经验）"></a>三、职业路线（5-10 岁经验）</h2><table>
|
|
<thead>
|
|
<tr>
|
|
<th>阶段</th>
|
|
<th>目标</th>
|
|
<th>关键步骤</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr>
|
|
<td><strong>技术负责人</strong></td>
|
|
<td>完成 <strong>3–5 个核心模块的技术方案</strong>。</td>
|
|
<td>按模块负责并交付。</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>架构师</strong></td>
|
|
<td>对整个平台制定 <strong>技术愿景、技术选型、非功能性指标</strong>。</td>
|
|
<td>1）制定技术路线图；2）做安全、数据治理、成本管控。</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>架构顾问</strong></td>
|
|
<td>对多家企业/业务线提供架构咨询。</td>
|
|
<td>参与技术评审、代码走查、培训。</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>高层 CxO（CTO/VP）</strong></td>
|
|
<td>负责技术战略、团队治理。</td>
|
|
<td>培养团队、构建技术生态。</td>
|
|
</tr>
|
|
</tbody></table>
|
|
<blockquote>
|
|
<p><strong>核心任务</strong></p>
|
|
<ul>
|
|
<li><em>选型</em>：在技术评估、成本评估、稳定性、社区支持等要素上做出决策。</li>
|
|
<li><em>方案落地</em>：把抽象设计写成 <strong>可执行的计划</strong>。</li>
|
|
<li><em>治理</em>：制定 <em>编码规范</em>、<em>CI/CD 标准</em>、<em>安全策略</em>、<em>成本追踪</em>。</li>
|
|
<li><em>治理工具</em>：使用 <strong>ArgoRollouts</strong>、<strong>Helm</strong>、<strong>OPA</strong>（Open Policy Agent）来实现自动化治理。</li>
|
|
</ul>
|
|
</blockquote>
|
|
<hr>
|
|
<h2 id="四、推荐路线图（时间轴示例）"><a href="#四、推荐路线图（时间轴示例）" class="headerlink" title="四、推荐路线图（时间轴示例）"></a>四、推荐路线图（时间轴示例）</h2><table>
|
|
<thead>
|
|
<tr>
|
|
<th>月份</th>
|
|
<th>阶段</th>
|
|
<th>关键任务</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr>
|
|
<td>1–6</td>
|
|
<td>Java 基础 & Spring Boot</td>
|
|
<td>1 个 CRUD 项目</td>
|
|
</tr>
|
|
<tr>
|
|
<td>7–12</td>
|
|
<td>中间件 & 微服务</td>
|
|
<td>3 个微服务 + 统一鉴权</td>
|
|
</tr>
|
|
<tr>
|
|
<td>13–18</td>
|
|
<td>系统设计 & DDD</td>
|
|
<td>业务拆解、领域模型</td>
|
|
</tr>
|
|
<tr>
|
|
<td>19–24</td>
|
|
<td>分布式 & 缓存</td>
|
|
<td>Kafka、Redis、分布式事务</td>
|
|
</tr>
|
|
<tr>
|
|
<td>25–36</td>
|
|
<td>云原生 & 运维</td>
|
|
<td>Kubernetes + Prometheus + Zipkin</td>
|
|
</tr>
|
|
<tr>
|
|
<td>37–48</td>
|
|
<td>设计模式 & 框架</td>
|
|
<td>事件驱动、Saga、CQRS</td>
|
|
</tr>
|
|
<tr>
|
|
<td>49–60</td>
|
|
<td>软技能 & 领导</td>
|
|
<td>组内技术分享、技术评审</td>
|
|
</tr>
|
|
<tr>
|
|
<td>60+</td>
|
|
<td>架构师</td>
|
|
<td>制定技术愿景、治理</td>
|
|
</tr>
|
|
</tbody></table>
|
|
<blockquote>
|
|
<p><strong>关键点</strong>：<strong>每个阶段都要有交付</strong>。写好<strong>技术文档、代码库、可复现的 demo</strong>，让别人能够看到你“解决了什么”而不是仅仅“看懂你说了些什么”。</p>
|
|
</blockquote>
|
|
<hr>
|
|
<h2 id="五、实战案例-经验总结"><a href="#五、实战案例-经验总结" class="headerlink" title="五、实战案例 / 经验总结"></a>五、实战案例 / 经验总结</h2><ol>
|
|
<li><strong>从业务痛点出发</strong><ul>
|
|
<li><strong>痛点</strong>：订单系统的并发高峰导致库存穿透。</li>
|
|
<li><strong>方案</strong>：采用 <strong>Redis 分布式锁 + 限流</strong>，把订单请求拆成两部分：下单 + 支付确认。</li>
|
|
<li><strong>结果</strong>：库存一致性恢复，系统 QPS 提升 200%。</li>
|
|
</ul>
|
|
</li>
|
|
<li><strong>选型要有成本意识</strong><ul>
|
|
<li>评估 <strong>AWS vs 阿里云</strong>：计算预计 8 万/年与 6 万/年，选阿里云。</li>
|
|
<li>在选型后做 <strong>性能基准</strong>（LTP测试）来验证业务水平。</li>
|
|
</ul>
|
|
</li>
|
|
<li><strong>保持“可审计”</strong><ul>
|
|
<li>在微服务中加入 <strong>OpenTelemetry</strong>，把日志、指标、追踪统一到 ELK。</li>
|
|
<li>业务日志采用 <strong>业务日志表</strong>，可以跟踪数据脏读与回滚路径。</li>
|
|
</ul>
|
|
</li>
|
|
<li><strong>持续学习</strong><ul>
|
|
<li>每周至少读 1 次技术文章（Hacker News、InfoQ、阿里巴巴技术分享）。</li>
|
|
<li>每年至少读 2 本系统架构书籍，把知识写进 <strong>技术笔记</strong>。</li>
|
|
</ul>
|
|
</li>
|
|
</ol>
|
|
<hr>
|
|
<h2 id="六、如何验证自己的能力？"><a href="#六、如何验证自己的能力？" class="headerlink" title="六、如何验证自己的能力？"></a>六、如何验证自己的能力？</h2><table>
|
|
<thead>
|
|
<tr>
|
|
<th>验证方式</th>
|
|
<th>说明</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr>
|
|
<td><strong>技术博客 / 公开课</strong></td>
|
|
<td>发布 8+ 篇系统设计技术文章，完成 1 次公开技术分享。</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>GitHub 项目</strong></td>
|
|
<td>至少 3 个真实业务项目或开源项目，PR 贡献。</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>项目验收记录</strong></td>
|
|
<td>给同事或客户交付至少 2 次业务改造。</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>面试评估</strong></td>
|
|
<td><strong>Top 技术平台</strong>（猎聘、BOSS 直聘）上，面试 “架构师” 级别。</td>
|
|
</tr>
|
|
<tr>
|
|
<td><strong>认证</strong></td>
|
|
<td><em>Oracle Certified Professional</em>、<em>AWS Certified Solutions Architect – Professional</em>、<em>阿里云架构师认证</em> 等。</td>
|
|
</tr>
|
|
</tbody></table>
|
|
<blockquote>
|
|
<p><strong>面试题准备</strong></p>
|
|
<ul>
|
|
<li>业务场景 → 需求分析 → 系统架构 → 数据模型 → 性能方案 → 监控与运维 → 失败模式与可恢复性</li>
|
|
<li>经典面试题：<ul>
|
|
<li>给定 <code>100 个并发用户</code> 同时下单，怎么防止库存穿透？</li>
|
|
<li>如何实现微服务之间的 <strong>幂等调用</strong>？</li>
|
|
<li>解释“CAP定理”并给出符合业务的解决方案。</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</blockquote>
|
|
<hr>
|
|
<h2 id="七、最后的实戰建議"><a href="#七、最后的实戰建議" class="headerlink" title="七、最后的实戰建議"></a>七、最后的实戰建議</h2><ol>
|
|
<li><strong>先学会 “怎么写代码”，再学 “怎么设计架构”。</strong><ul>
|
|
<li>把代码质量放在首位：单元测试、代码评审、CI/CD。</li>
|
|
</ul>
|
|
</li>
|
|
<li><strong>与业务同事保持紧密沟通</strong>，避免 “技术孤岛”。</li>
|
|
<li><strong>学习</strong>：架构师不只是技术，更是 <strong>业务、合规、成本</strong> 的统筹者。</li>
|
|
<li><strong>保持好奇心</strong>：阅读每季度的 <strong>技术白皮书</strong>（阿里巴巴、腾讯、Google、Microsoft）。</li>
|
|
<li><strong>培养 “反脆弱” 思维</strong>：系统设计不是一次性完成的，而是 <strong>持续演进</strong>。</li>
|
|
<li><strong>分享</strong>：通过技术分享会、博客或社区演讲，把自己的知识体系和思维体系固化。</li>
|
|
</ol>
|
|
<hr>
|
|
<blockquote>
|
|
<p><strong>结语</strong><br>成为 Java 架构师是一段 <strong>从 “熟练编码” → “系统设计” → “业务与技术双栖”</strong> 的成长旅程。只要你把每一步拆解成可执行的目标，系统练习并且用落地项目检验自己的能力，就<br>能稳扎稳打地迈向架构师角色。祝你早日成为团队里的架构引路人！</p>
|
|
</blockquote>
|
|
<p>如果你想进一步细化任何一板块（如某个具体的技术选型、微服务拆分方案，或是面试技巧），随时告诉我！祝你编码愉快 🚀✌️</p>
|
|
<blockquote>
|
|
<p>内容正确性仅供参考<br>只是输出的质量体感可能赶得上最初版的chatgpt了，按这种效果本地用来做一些辅助工作的能力感觉是够用了。根据官方的测评20b的模型也是超过了o3-mini，主要是目前的内存占用还比较吃力对我来说，如果是32g内存的话应该就比较舒服了，也期待会有更小的且效果不输于gpt-oss:20b的模型，本地的模型使用就会越来越普遍且好用</p>
|
|
</blockquote>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>nas 中使用 tmm 刮削视频</title>
|
|
<url>/2023/07/02/%E4%BD%BF%E7%94%A8-tmm-%E5%88%AE%E5%89%8A%E8%A7%86%E9%A2%91/</url>
|
|
<content><![CDATA[<p>最近折腾了个自建的 nas，为了使用 jellyfin 这样的影视应用需要对视频进行刮削，对于电视剧来说还是有些不一样的，<br>比如我要刮削这部经典电视剧纪晓岚<br><img data-src="https://img.nicksxs.me/blog/gzJIWi.png"><br>像这样的命名方式在 tmm 中是无法识别的，或者就得一集一集进行制定刮削，<br><img data-src="https://img.nicksxs.me/blog/S5UtuM.png"><br>所以第一步需要进行改名<br>比如这是第一季的，那就是 S01，然后按集数 E01，第一季第一集的文件名就是 S01E01.mkv<br>然后右键点击搜索刮削，默认会以文件夹名进行搜索，是在 tmdb 数据库进行搜索<br><img data-src="https://img.nicksxs.me/blog/hExmO7.png"><br>这样除非文件夹名很符合要求，一般都刮削不出来，所以需要有两种刮削方式，一种就是比较标准的命名，这样的名字可以手动先去豆瓣或者 tmdb 搜索，另一个种也可以在豆瓣找到 imdb 的 id 进行搜索<br><img data-src="https://img.nicksxs.me/blog/gUhZNC.png"><br>但有时候也会搜不到，比如这个纪晓岚就是搜不到，但是之前的一些韩剧什么的很多都能搜到<br><img data-src="https://img.nicksxs.me/blog/hN7XEL.png"><br>这个其实是最基本的刮削，但是这样刮削了需要在 jellyfin 那取消元数据下载<br><img data-src="https://img.nicksxs.me/blog/4oXenN.png">，这样就不会被覆盖</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>nas</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>nas</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>使用 LM Studio 在本地部署 Deepseek-R1 的蒸馏版大模型</title>
|
|
<url>/2025/02/23/%E4%BD%BF%E7%94%A8-LM-Studio-%E5%9C%A8%E6%9C%AC%E5%9C%B0%E9%83%A8%E7%BD%B2-Deepseek-%E5%A4%A7%E6%A8%A1%E5%9E%8B/</url>
|
|
<content><![CDATA[<p>deepseek-v3 和 deepseek-r1 是由深度求索公司推出的大语言模型，并且在生成效果上基本已经到达国内甚至全球的顶级水准，并且是免费使用，并且是开源的<br>连着用三个并且不是我文字能力差，而是真的佩服deepseek这家公司和创始人梁文峰，作为校友感觉是真的忝列门墙<br>早先了解的幻方量化就是一家小众低调但又多金的公司，奈何都是招的全栈工程师，可能不是太适合，当前对于deepseek来说那就更遥远了，从公开的不一定可靠的信息来看，目前在deepseek的都是像清北的博士这样的顶级人才，而且在人工智能这块，只是浅显地涉及过一些机器学习相关的工作，对于最前沿的大模型这块着实还有比较大的差距，当然也是在学习中。<br>虽然不是完全专业，但是对比大部分跟这个领域完全不相关的人来说，可能还是可以给出一些半专业的见解<br>首先例如 deepseek 只使用了很少的显卡来训练这一点也只是相对的，比如最新出的 grok 3 的20万卡，的确省了比较多资源，使用了2048张H800显卡，说实话这个也不是任何公司可以负担得起的，训练的总成本估计是560万美元，H800的价格预计在20万-40万元，后期因为禁售等可能涨价至后者，即使以20万一张的价格，光显卡的价格就需要4亿的资金，这不是随便一个公司就负担得起的，也得亏之前幻方就是靠机器学习做量化交易，原先就有显卡储备，并且盈利能力很强。<br>第二点是对于现在市面上乱七八糟的 deepseek 服务，因为实在太火了，导致 deepseek 的官方服务一直处于限流状态，然后就出现了各种说上线了 deepseek 大模型，以真实情况来讲除了少数几家本身就在做这个的，浑水摸鱼的存在比较多，例如用了 deepseek 的 api 接口的，或者是类似于我后面要讲的，只是部署了个蒸馏模型，因为全尺寸模型，再不用什么垃圾佬的省显卡黑科技的情况，671B的大小我通过 grok-3 的推理计算大概也需要 2000张H800显卡才能支持约100qps，这个可能也不是随便一家公司能负担得起的，并且还有这个带来的巨大能耗费用，只是把它跑起来因为模型文件有接近700G，那么至少需要 80G 现存的 H800 显卡 9张，为了推理效率更高，那要求就更高了。<br>而对于我们本地想使用这个模型的话，普通情况下也只能使用小尺寸的蒸馏后模型，这个效果肯定是跟deepseek官方和比如腾讯等大厂提供的全尺寸是没法比的，所以很多网上的视频和文章说教你本地部署 deepseek-r1 的绝大多数是这种蒸馏模型，因为普通人哪怕是最新的5090显卡也只有32G的显存，不可能载入全尺寸模型，按多卡算得 22 张 5090，我相信普通人很难搞到这么多显卡，并且常规的 U 有PCIE通道数量上线，单机不太可能能承载22张 5090。<br>而全尺寸的671B的模型跟常规家用电脑能跑起来的例如 7B 跟 14B 的差异类似于视频，480P的视频跟4K的视频，虽然都能看，但是效果来说还是差挺多，所以有些同学觉得官方的 deepseek 经常繁忙，而听信网上一堆文章说能直接在自己电脑上跑 deepseek-r1 的，可能还是以忽悠为主。本地自己的家用电脑部署主要是能够方便的使用例如 deepseek-r1 的小参数量蒸馏版本，来提供一些非高标准的要求，以及一些学习目的，比如我们可以在本地部署一个 7b 或者 14b 版本的模型，然后再基于这个模型加上 RAG 功能，这样就能把我们平常躺在那的笔记跟知识库变成一个能够更高效的提供知识检索的高级知识库。<br>这里我们使用LM Studio 在Windows 环境下部署一个 7b的蒸馏模型，比如 <code>lmstudio-community/DeepSeek-R1-Distill-Qwen-7B-GGUF/DeepSeek-R1-Distill-Qwen-7B-Q4_K_M.gguf</code> 这个模型，<br>首先这个模型是基于 <code>Qwen2.5-Math-7B</code> 通过监督微调（SFT）从 DeepSeek-R1 生成的数据集上训练得到的。 所以基础模型来说它是有目的性的，主要为数学方向做的训练，在此基础上会学习到 <code>deepseek-r1</code> 的推理能力<br>此前我们写过使用 ollama 来在本地跑谷歌开源的 Gemma 模型，因为 deepseek-r1 的这些蒸馏模型在ollama的支持没那么快，所以我是先用lm studio来跑了<br><a href="https://lmstudio.ai/">Lm Studio</a> 相比 ollama 来说是带有了 UI 界面的一个大模型运行软件，这样就更方便的可以开箱即用<br>从官网下载完安装完对应平台的 LM Studio 以后差不多是这样子, 我以Win10系统为例<br><img data-src="https://img.nicksxs.me/blog/%E5%BE%AE%E4%BF%A1%E6%88%AA%E5%9B%BE_20250223201814.png"><br>默认还没有任何模型，我们可以点击左侧的搜索样图标<br><img data-src="https://img.nicksxs.me/blog/%E5%BE%AE%E4%BF%A1%E6%88%AA%E5%9B%BE_20250223202245.png"><br>在这里可以进行模型的搜索，但是由于众所周知的原因，这个搜索模型的并不像ollama一样能够很顺畅地下载，一般国内网络是打不开的，或者说只有模型列表，但是无法打开详情页面，我们需要一些小操作<br>首先我们找到LM Studio的安装目录，例如我这样的 <code>D:\Program Files\LM Studio</code> 我的是自己修改过的，常规的方式可以在桌面 LM Studio 图标上右键，然后点击 “打开文件所在的位置” 就可以找到LM Studio 的安装目录了，然后继续打开安装目录中的 resources 文件夹，再打开其中的 app 目录，就可以看到一个 <code>.webpack</code> 文件夹，建议用 vscode 或者sublime 打开目录，然后全局搜索 <code>huggingface.co</code> 替换成镜像地址 <code>hf-mirror.com</code>，然后重新打开 <code>LM Studio</code> 就可以搜索到上面提到的模型了，可能是由于这个镜像站没那么充足的资源，一般是白天的下载速度会快一点，晚上一般速度不太行，有条件的可以上全局代理，不过幸好 LM Studio 是支持断点续传的，只是它的下载很容易超时后自动异常，并不会自动重试，，这是个比较头疼的地方。<br>下载完成后就可以在顶部选择模型<br><img data-src="https://img.nicksxs.me/blog/%E5%BE%AE%E4%BF%A1%E6%88%AA%E5%9B%BE_20250223205634.png"><br>这里会有一些配置，比如显卡的超载情况，是否将模型完整载入内容，上下文长度，显卡我这边是笔记本上的 3060，只有6g的显存，所以用不了很大的模型，内存倒可以载入，但是除非只用cpu来运行，否则内存载入只是将从硬盘载入显存的速度加快了，因为普通的PC并不像Mac使用的是统一内存，当然目前好像有清华的团队已经搞出来使用kTransformer通过一张4090显卡加384G内存运行全尺寸的 <code>deepseek-r1</code> ，这类暂时不考虑在内，如果只是一些简单的使用场景，可以把上下文的大小调小一些，这样可以加快生成速度<br>问一个比较简单的问题，也是我比较讨厌的问题<br><code>9.11跟9.08 哪个大</code><br><img data-src="https://img.nicksxs.me/blog/%E5%BE%AE%E4%BF%A1%E6%88%AA%E5%9B%BE_20250223210610.png"><br><img data-src="https://img.nicksxs.me/blog/%E5%BE%AE%E4%BF%A1%E6%88%AA%E5%9B%BE_20250223210628.png"><br>果然是以数学模型蒸馏出来的，其实很多用来话题讨论度比较高的用来比较大模型能力水平的都是比较没意义的，因为这种问题不依赖于大模型，甚至都不用搜索就能知道，会让这么大成本训练出来的大模型显得更加弱智，本身就不是为了这个而生的，唠叨的比较多，后面可以继续讲</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>使用 chatbox 来连接火山引擎等api服务来使用Deepseek-R1 全尺寸大模型</title>
|
|
<url>/2025/03/09/%E4%BD%BF%E7%94%A8-chatbox-%E6%9D%A5%E8%BF%9E%E6%8E%A5%E7%81%AB%E5%B1%B1%E5%BC%95%E6%93%8E%E7%AD%89api%E6%9C%8D%E5%8A%A1%E6%9D%A5%E4%BD%BF%E7%94%A8Deepseek-R1-%E5%85%A8%E5%B0%BA%E5%AF%B8%E5%A4%A7%E6%A8%A1%E5%9E%8B/</url>
|
|
<content><![CDATA[<p>chatbox 是个可以链接大模型api服务的客户端工具，之前我们说的都是使用自己部署的蒸馏模型，如果现在本地使用全尺寸大模型的话，就可以使用像 chatbox 这样的客户端工具<br>首先我们需要下载 chatbox 这个软件，注意到现在他们家的网站需要科学上网才能打开，所以需要处理下网络问题下载完成后我们需要去注册并开通大模型的api服务<br>目前有像火山引擎，硅基流动等等在提供支持deepseek-r1全尺寸模型的api服务，可以自由选择，这里以火山引擎为例<br>首先是注册火山引擎，地址是这 <a href="https://www.volcengine.com/experience/ark">https://www.volcengine.com/experience/ark</a><br><img data-src="https://img.nicksxs.me/blog/D26qHi.png"><br>然后不介意的话，可以填下我的邀请码，我们都可以获得比较可观数量的token额度，我的邀请码是 7R4RJOO8<br>接下去就是开通api服务，<br><img data-src="https://img.nicksxs.me/blog/cLgVNU.png"><br>这里就要使用我们自定义的模型提供上，这里的api域名是参考<br><img data-src="https://img.nicksxs.me/blog/6DgvEj.png"> 的url进行配置，以及确定模型名是 deepseek-r1-250120<br><img data-src="https://img.nicksxs.me/blog/Nq63sm.png"><br>接下去我们就可以在chatbox中使用我们的deepseek-r1大模型了，<br>比如我让它生成一个网页版的贪吃蛇<br><img data-src="https://img.nicksxs.me/blog/5xkJXp.png"><br>还是能够很流畅的响应，毕竟火山背靠头条<br>这边也贴一下我的火山邀请码<br><img data-src="https://img.nicksxs.me/blog/%E7%81%AB%E5%B1%B1%E5%BC%95%E6%93%8E%E9%82%80%E8%AF%B7%E6%B5%B7%E6%8A%A5.png"><br>另外还有硅基流动的邀请码 idw8RreO 同样我们都能获得一定数量的token<br><img data-src="https://img.nicksxs.me/blog/%E7%A1%85%E5%9F%BA%E6%B5%81%E5%8A%A8.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>关于 npe 的一个小记忆点</title>
|
|
<url>/2023/07/16/%E5%85%B3%E4%BA%8E-npe-%E7%9A%84%E4%B8%80%E4%B8%AA%E5%B0%8F%E8%AE%B0%E5%BF%86%E7%82%B9/</url>
|
|
<content><![CDATA[<p>Java 中最常见的一类问题或者说异常就是 <code>NullPointerException</code>，而往往这种异常我们在查看日志的时候都是根据打印出来的异常堆栈找到对应的代码以确定问题，但是有一种情况会出现一个比较奇怪的情况，虽然我们在日志中打印了异常</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">log.error(<span class="string">"something error"</span>, e);</span><br></pre></td></tr></table></figure>
|
|
<p>这样是可以打印出来日志，譬如 slf4j 就可以把异常作为最后一个参数传入</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">void</span> <span class="title function_">error</span><span class="params">(String var1, Throwable var2)</span>;</span><br></pre></td></tr></table></figure>
|
|
<p>有的时候如果不小心把异常通过占位符传入就可能出现异常被吞的情况，不过这点不是这次的重点<br>即使我们正常作为最后一个参数传入了，也会出现只打印出来</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">something error, java.lang.NullPointerException: <span class="literal">null</span></span><br></pre></td></tr></table></figure>
|
|
<p>这样就导致了如果我们这个日志所代表的异常包含的代码范围比较大的话，就不确定具体是哪一行出现的异常，除非对这些代码逻辑非常清楚，这个是为什么呢，其实这也是 jvm 的一种优化机制，像我们目前主要还在用的 jdk8中使用的 jvm 虚拟机 hotspot 的一个热点异常机制，对于这样一直出现的相同异常，就认为是热点异常，后面只会打印异常名，不再打印异常堆栈，这样对于 io 性能能够提供一定的优化保证，这个可以通过 jvm 启动参数<br><code>-XX:-OmitStackTraceInFastThrow</code> 来关闭优化，这个问题其实在一开始造成了一定困扰，找不准具体是哪一行代码，不过在知道这个之后简单的重启也能暂时解决问题。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>体验最新的gemini cli工具来实现一个简单的网页小程序</title>
|
|
<url>/2025/06/29/%E4%BD%93%E9%AA%8C%E6%9C%80%E6%96%B0%E7%9A%84gemini-cli%E5%B7%A5%E5%85%B7%E6%9D%A5%E5%AE%9E%E7%8E%B0%E4%B8%80%E4%B8%AA%E7%AE%80%E5%8D%95%E7%9A%84%E7%BD%91%E9%A1%B5%E5%B0%8F%E7%A8%8B%E5%BA%8F/</url>
|
|
<content><![CDATA[<p>谷歌新出了个对标claude code的命令行工具，叫gemini-cli，还是开源的，所以赶紧来体验下，就是前提是有科学上网，要去google的ai studio申请gemini的api key<br>然后在有科学网络的情况下用node 18及以上的版本安装</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">npm install -g @google/gemini-cli</span><br></pre></td></tr></table></figure>
|
|
<p>然后在环境变量里设置gemini key</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">export GEMINI_API_KEY="xxxxxxxx"</span><br></pre></td></tr></table></figure>
|
|
<p>接着运行<code>gemini</code>就可以啦<br>这里我是刚好有个需求，因为娃最近有点不太爱喝奶，经常剩下，想要记录统计下，看看每天的总的奶量<br>那我就建了个空的目录，并在这个目录下运行了 <code>gemini</code> ，然后输入了我的需求，<br>原始的诉求是这样<br><code>我想要写个网页版小程序，可以按时间记录宝宝每次的奶量，包括泡了多少，剩了多少，能按天统计每天的奶量，以及查看趋势</code><br>然后gemini就开始运行</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">✦ 好的，这个需求很明确，我们可以创建一个名为“奶量备忘” (Milk Memo) 的网页应用来实现它。</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> 这是一个很好的项目，完全可以通过前端技术实现，将数据存储在浏览器中，无需复杂的后端。</span><br><span class="line"></span><br><span class="line"> 这是我的计划：</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> 应用名称：奶量备忘 (Milk Memo)</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> 核心功能：</span><br><span class="line"> 1. 记录喂奶量：一个简洁的表单，用于输入“冲泡量”和“剩余量”。提交后，应用会自动计算“实际饮用量”并记录当前时间。</span><br><span class="line"> 2. 每日记录列表：清晰地展示当天的所有喂奶记录，包括时间、冲泡量、剩余量和实际饮用量。</span><br><span class="line"> 3. 今日统计：在页面顶部或底部显眼地展示当天累计的总饮用量。</span><br><span class="line"> 4. 历史趋势：一个图表页面，用柱状图或折线图展示最近7天或30天的每日总饮用量，帮助您直观地了解宝宝的饮奶趋势。</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> 技术选型：</span><br><span class="line"> * 前端框架: React - 用于构建用户界面，组件化的思想很适合这个应用。</span><br><span class="line"> * 样式与布局: Bootstrap - 一个流行的CSS框架，可以快速构建出简洁、美观且移动端优先的响应式布局。</span><br><span class="line"> * 图表: Chart.js - 一个强大的图表库，用于实现“历史趋势”功能。</span><br><span class="line"> * 数据存储: 浏览器本地存储 (Local Storage) - 无需后端和数据库，所有数据将安全地存储在您自己的浏览器里，简单快捷。</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> 用户体验与设计：</span><br><span class="line"> 我将采用现代化、简洁的设计风格，确保在手机浏览器上也能获得良好的操作体验。界面会以卡片的形式展示数据，按钮和输入框都会清晰易用。</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> 这个计划听起来如何？如果同意，我将开始着手创建这个应用。</span><br></pre></td></tr></table></figure>
|
|
<p>然后我输入了同意他就开始帮我实现了<br>它会请求我的同意来运行比如npm命令，安装依赖等等<br>然后就自己运行了<code>npm start</code> 来演示运行这个应用<br><img data-src="https://img.nicksxs.me/milk_memo_v1.png"><br>不算非常有设计感，但是基本的功能是已经实现了<br>接着我又让它帮我增加了时间选项，以及趋势图查看功能,<br>接着它就又生成了一版</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">✦ 功能已经更新完毕。</span><br><span class="line"></span><br><span class="line"> 现在应用具备了以下功能：</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> * 自定义时间：您可以在添加记录时，精确选择到分钟。</span><br><span class="line"> * 视图切换：通过顶部的按钮，可以在“记录列表”和“历史趋势”图表之间自由切换。</span><br><span class="line"> * 趋势图表：在“历史趋势”视图中，您可以看到一个折线图，展示了每天的总饮奶量。</span><br><span class="line"> * 数据排序：所有记录现在会按时间从近到远排序，方便查看最近的条目。</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/milk_memo_v2.png"><br>感觉完成了所需的95%的功能，除了在设计上还能再美观些，以及增加一些服务端存储数据的功能，基本是个比较完善的，我自己都完全没介入，的确ai的发展真的一日千里，另外我还是比较看好谷歌的，没有什么大新闻，就一直努力追赶，现在基本已经是第一梯队了，很好很强大</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>使用rclone来当临时图床</title>
|
|
<url>/2025/05/11/%E4%BD%BF%E7%94%A8rclone%E6%9D%A5%E5%BD%93%E4%B8%B4%E6%97%B6%E5%9B%BE%E5%BA%8A/</url>
|
|
<content><![CDATA[<p>鉴于用upic配置r2存储的图床失败后，目前的方式退化了一下，直接使用rclone作为临时替代<br>但是也才了个小坑，因为rclone在做copy的时候是不管bucket是否存在的，直接会调用 CreateBucket 命令<br>同时我们的accessKey一般是只有对bucket内部文件的读写权限，所以会导致一个比较尴尬的问题<br>直接用copy就是报</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">2025/05/11 21:16:04 NOTICE: Failed to copy: failed to prepare upload: operation error S3: CreateBucket, https response error StatusCode: 403, RequestID: , HostID: , api error AccessDenied: Access Denied</span><br></pre></td></tr></table></figure>
|
|
<p>这个403的错误，结果谷歌搜索了下<br>可以通过这个选项进行屏蔽</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">--s3-no-check-bucket</span><br></pre></td></tr></table></figure>
|
|
<p>这样我就能用 </p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">./rclone --s3-no-check-bucket copy ~/Downloads/r2/test_for_rclone.png r2_new:blog</span><br></pre></td></tr></table></figure>
|
|
<p>来上传了，只是这里相比uPic麻烦了两步，第一截图后需要对文件进行命名，不然就是qq截图或者微信截图的文件名，<br>第二上传后需要自己拼图片的链接，当然希望是能够使用uPic的，但目前的问题是uPic似乎对于s3协议的支持不那么好<br>噢，对了，我用的是直接基于开源代码的自己打包版本，个人感觉还有点不太习惯从免费用成付费软件，主要是它是开源的<br>打算空了再折腾打包试试，或者自己调试下，实在不行就付费了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>体验</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>恶意盗刷</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>关于公共交通再吐个槽</title>
|
|
<url>/2021/03/21/%E5%85%B3%E4%BA%8E%E5%85%AC%E5%85%B1%E4%BA%A4%E9%80%9A%E5%86%8D%E5%90%90%E4%B8%AA%E6%A7%BD/</url>
|
|
<content><![CDATA[<p>事情源于周末来回家发生的两件事情，先是回去的时候从高铁下车要坐公交，现在算是有个比较好的临时候车点了，但是可能由于疫情好转，晚上都不用检查健康码就可以进候车点，但是上公交的时候还是需要看健康码，一般情况下从高铁下来的，各个地方的人都有，而且也不太清楚这边上公交车需要查验健康码，我的一个看法是候车的时候就应该放个横幅告示，或者再配合喇叭循环播放，请提前准备好健康码，上车前需要查验，因为像这周的情况，我乘坐的那辆车是间隔时间比较长，而且终点是工业开发区，可能是比较多外来务工人员的目的地，正好这部分人可能对于操作手机检验健康码这个事情不太熟悉，所以结果就是头上几个不知道怎么搞出来健康码，然后让几乎有满满一整车的人在后面堵着，司机又非常厌烦那些没有提前出示健康码的，有位乘客搞得久了点，还被误以为没刷卡买票，差点吵起来，其实公共交通或者高铁站负责的在公交指引路线上多写一句上车前需要查验健康码，可能就能改善比较多，还有就是那个积水的路，这个吐槽起来就一大坨了，整个绍兴像 dayuejin 一样到处都是破路。<br>第二个就是来杭州的时候，经过人行横道，远处车道的公交车停下来等我们了，为了少添麻烦总想快点穿过去，但是这时靠近我们的车道（晚上光线较暗，可见度不佳）有一辆从远处来的奥迪 A4 还是 A5 这种的车反而想加速冲过去，如果少看一下可能我已经残了，交规考的靠近人行道要减速好像基本都是个摆设了，杭州也只有公交车因为一些考核指标原因会主动礼让，人其实需要有同理心，虽然可能有些人是开车多于骑车走路的，但是总也不可能永远不穿人行道吧，甚至这些人可能还会在人行道红灯的时候走过去。这个事情不是吐槽公共交通的，只是也有些许关系，想起来还有一件事也是刚才来杭州的时候看到的，等公交的时候看到有辆路虎要加塞，而目标车道刚好是辆大货车，大货车看到按了喇叭，路虎犹豫了下还是挤进去了，可能是对路虎的扛撞性能非常自信吧，反正我是挺后怕的，这种级别的车，被撞了的话估计还是鸡蛋撞石头，吨位惯性在那，这里再延伸下，挺多开豪车的人好像都觉得这路上路权更大一些，谁谁都得让着他，可能实际吃亏的不多，所以越加巩固了这种思维，当真的碰到不管的可能就会明白了，路权这个事情在天朝也基本没啥人重视，也没想说个结论，就到这吧</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>公交</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>开车</tag>
|
|
<tag>加塞</tag>
|
|
<tag>糟心事</tag>
|
|
<tag>规则</tag>
|
|
<tag>公交</tag>
|
|
<tag>路政规划</tag>
|
|
<tag>基础设施</tag>
|
|
<tag>杭州</tag>
|
|
<tag>健康码</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>关于读书打卡与分享</title>
|
|
<url>/2021/02/07/%E5%85%B3%E4%BA%8E%E8%AF%BB%E4%B9%A6%E6%89%93%E5%8D%A1%E4%B8%8E%E5%88%86%E4%BA%AB/</url>
|
|
<content><![CDATA[<p>最近群里大佬发起了一个读书打卡活动，需要每天读一会书，在群里打卡分享感悟，争取一个月能读完一本书，说实话一天十分钟的读书时间倒是问题不大，不过每天都要打卡，而且一个月要读完一本书，其实难度还是有点大的，不过也想试试看。<br>之前某某老大给自己立了个 flag，说要读一百本书，这对我来说挺难实现的，一则我也不喜欢书只读一小半，二则感觉对于喜欢看的内容范围还是比较有限制，可能也算是比较矫情，不爱追热门的各类东西，因为往往会有一些跟大众不一致的观点看法，显得格格不入。所以还是这个打卡活动可能会比较适合我，书是人类进步的阶梯。<br>到现在是打卡了三天了，读的主要是白岩松的《幸福了吗》，对于白岩松，我们这一代人是比较熟悉，并且整体印象比较不错的一个央视主持人，从《焦点访谈》开始，到疫情期间的各类一线节目，可能对我来说是个三观比较正，敢于说一些真话的主持人，这中间其实是有个空档期，没怎么看电视，也不太关注了，只是在疫情期间的节目，还是一如既往地给人一种可靠的感觉，正好有一次偶然微信读书推荐了白岩松的这本书，就看了一部分，正好这次继续往下看，因为相对来讲不会很晦涩，并且从这位知名央视主持人的角度分享他的过往和想法看法，还是比较有意思的。<br>从对汶川地震，08 年奥运等往事的回忆和一些细节的呈现，也让我了解比较多当时所不知道的，特别是汶川地震，那时的我还在读高中，真的是看着电视，作为“猛男”都忍不住泪目了，共和国之殇，多难兴邦，但是这对于当事人来说，都是一场醒不过来的噩梦。<br>然后是对于足球的热爱，其实光这个就能掰扯很多，因为我不爱足球，只爱篮球，其中原因有的没的也挺多可以说的，但是看了他的书，才能比较深入的了解一个足球迷，对足球，对中国足球，对世界杯，对阿根廷的感情。<br>接下去还是想能继续坚持下去，加油！</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>读后感</category>
|
|
<category>白岩松</category>
|
|
<category>幸福了吗</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>读后感</tag>
|
|
<tag>读书</tag>
|
|
<tag>打卡</tag>
|
|
<tag>幸福了吗</tag>
|
|
<tag>足球</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>分享一次折腾老旧笔记本的体验-续篇</title>
|
|
<url>/2023/02/12/%E5%88%86%E4%BA%AB%E4%B8%80%E6%AC%A1%E6%8A%98%E8%85%BE%E8%80%81%E6%97%A7%E7%AC%94%E8%AE%B0%E6%9C%AC%E7%9A%84%E4%BD%93%E9%AA%8C-%E7%BB%AD%E7%AF%87/</url>
|
|
<content><![CDATA[<p>接着上一篇的折腾记，因为这周又尝试了一些新的措施和方法，想继续记录分享下，上周的整体情况大概是 Ubuntu 系统能进去了，但是 Windows 进不去，PE 也进不去，Windows 启动盘也进不去，因为我的机器加过一个 msata 的固态，Windows 是装在 msata 固态硬盘里的，Ubuntu 是装在机械硬盘里的，所以有了一种猜测就是可能这个固态硬盘有点问题，还有就是还是怀疑内存的问题，正好家里还有个msata 的固态硬盘，是以前想给LD 的旧笔记本换上的，因为买回来放在那没有及时装，后面会又找不到，直到很后面才找到，LD 也不怎么用那个笔记本了，所以就一直放着了，这次我就想拿来换上。<br>周末回家我就开始尝试了，换上了新的固态硬盘后，插上 Windows 启动 U 盘，这次一开始看起来有点顺利，在 BIOS 选择 U 盘启动，进入了 Windows 安装界面，但是装到一半，后面重启了之后就一直说硬盘有问题，让重启，但是重启并没有解决问题，变成了一直无效地重复重启，再想进 U 盘启动，就又进不去了，这时候怎么说呢，感觉硬盘不是没有问题，但是呢，问题应该不完全出在这，所以按着总体的逻辑来讲，主板带着cpu 跟显卡，都换掉了，硬盘也换掉了，剩下的就是内存了，可是内存我也尝试过把后面加的那条金士顿拔掉，可还是一样，也尝试过用橡皮擦金手指，这里感觉也很奇怪了，找了一圈了都感觉没啥明确的原因，比如其实我的猜测，主板电池的问题，一些电阻坏掉了，但是主板是换过的，那如果内存有问题，照理我用原装的那条应该会没问题，也有一种非常小的可能，就是两条内存都坏了，或者说这也是一种不太可能的可能，所以最后的办法就打算试试把两条内存都换掉，不过现在网上都找不到这个内存的确切信息了，只能根据大致的型号去买来试试，就怕买来的还是坏的，其实也怕是这个买来的主板因为也是别的拆机下来的，不一定保证完全没问题，要是有类似的问题或者也有别的问题导致开不起来就很尴尬，也没有很多专业的仪器可以排查原因，比如主板有没有什么短路的，对了还有一个就是电源问题，但是电源的问题也就可能是从充电器插的口子到主板的连线，因为 LD 的电源跟我的口子一样，也试过，但是结果还是一样，顺着正常逻辑排查，目前也没有剩下很明确的方向了，只能再尝试下看看。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Windows</category>
|
|
<category>小技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Windows</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>分享一次折腾老旧笔记本的体验-续续篇</title>
|
|
<url>/2023/02/26/%E5%88%86%E4%BA%AB%E4%B8%80%E6%AC%A1%E6%8A%98%E8%85%BE%E8%80%81%E6%97%A7%E7%AC%94%E8%AE%B0%E6%9C%AC%E7%9A%84%E4%BD%93%E9%AA%8C-%E7%BB%AD%E7%BB%AD%E7%AF%87/</url>
|
|
<content><![CDATA[<p>上周因为一些事情没回去，买好了内存条这周才回去用，结果不知道是 U 盘的问题还是什么其他原因原来装的那个系统起不来，然后想用之前一直用的 LD 的笔记本，结果 LD 的老笔记本也起不来了，一直卡在启动界面，猜测可能是本身也比较老了，用的机械硬盘会不会被我一直动而损坏了，所以就想搞个老毛桃跟新搞一个启动盘，结果这周最大的坑就在这了，在我的 15 寸 mbp 上用etcher做了个 win10 的启动盘，可能是分区表格式是 GUID，不是 MBR 的，老电脑不能识别，就像装个虚拟机来做启动盘，结果像老毛桃不支持这种用虚拟机的，需要本地磁盘，然后我就有用 bootcamp 安装了个 win10，又没网络，查了下网络需要用 bootcamp 下载 Windows Support 软件，下了好久一直没下完，后来又提示连不上苹果服务器，好不容易下完了，开启 win10 后，安装到一半就蓝屏了，真的是够操蛋的，在这个时候真的很需要有一个 Windows 机器的支持，LD 的那个笔记本也跟我差不多老了，虽然比较慢但之前好歹也还可以用，结果这下也被我弄坏了，家里就没有其他机器可以支持了，有点巧妇难为无米之炊的赶脚，而且还是想再折腾下我的老电脑，后面看看打算做几个 U 盘，分工职责明确，一个老毛桃的PE 盘，老毛桃的虽然不是很纯净的，但是是我一直以来用过内置工具比较全，修复启动功能比较强大的一个 pe，没用过 winpe，后续可以考虑使用下，一个 win7 的启动盘，因为是老电脑，如果后面是新的电脑可以考虑是 win10，一个 win to go 盘，这样我的 mbp 就可以直接用 win10 了，不过想来也还是要有一台 win 机器才比较好，不然很多时候都是在折腾工具，工欲善其事必先利其器，后面也可以考虑能够更深入的去了解主板的一些问题，我原来的那个主板可能是一些小的电阻等问题，如果能自己学习解决就更好了，有时候研究下还是有意思的。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Windows</category>
|
|
<category>小技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Windows</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>分享一次折腾老旧笔记本的体验</title>
|
|
<url>/2023/02/05/%E5%88%86%E4%BA%AB%E4%B8%80%E6%AC%A1%E6%8A%98%E8%85%BE%E8%80%81%E6%97%A7%E7%AC%94%E8%AE%B0%E6%9C%AC%E7%9A%84%E4%BD%93%E9%AA%8C/</url>
|
|
<content><![CDATA[<p>上大学的时候买了第一个笔记本，是联想的小y，y460，配置应该是 i5+2g 内存，500g 硬盘，ati 5650 的显卡，那时候还是比较时髦的带有集显和独显切换的，一些人也觉得它算是一代“神机”，陪我度过了大学四年，还有毕业后的第一年，记得中间还用的比较曲折，差不多第一学期末的时候硬盘感觉有时候会文件操作卡住，去售后看了下，果然硬盘出问题了，还在当场把硬盘的东西都拷贝到新买的移动硬盘里，不过联想的售后还是比较不错的，在保内可以直接免费换，如果出保了估计就要修不起了，后面出了个很大的问题，就是屏幕花屏了，而且它的花屏是横着的会有那种上升的变化，一开始感觉跟湿度还有温度有点关系，天气冷了就很容易出现，天气热了就好一点，出现的概率小一点，而且是合盖后再开会好，过一会又会上升，只是这么度过了保修期，去售后问了下，修一下要两千多，后来是在玉泉学校里面的维修店，好像是花了六百多，省是省了很多，但是后面使用的时候发现有点漏光，而且两三个亮点，总归还是给我那不富裕的经济条件带来了一个正常屏幕，不过是挺影响使用，严格来说都没法用了，后面基本是大半个屏幕花掉，接近整个屏幕花掉，至此大学就是这么用过去了。<br>噢对了，中间在大二的时候加了一条 2g 的内存，因为像操作系统课需要用虚拟机，2g 内存不太够，不过那会用的都是 32 位的 win7 系统，实际使用用不到 4g 内存，得使用破解补丁才能识别到，后来在大学毕业后由于收入也低，想换其他电脑，特别是 mac，买不起，电脑在那会还会玩玩 DNF，但是风扇声很大，也想优化下，就买了个 msata 的固态硬盘，因为刚好有个口子留着，在那之前一直搜到的是把光驱位拆掉换上 sata 固态硬盘，这对于那会的我来说操作难度实在有点大，换上了msata 固态之后还是有比较大的提升的，把操作系统装到固态硬盘后其他盘就只放数据了，后面还装过黑苹果，只是不小心被室友强制关机了，之后就起不来了，也不知道啥原因，后续想继续按原来的操作装，也没成功，再往后就配了一台台式机，这个笔记本就放在那没啥用了，后面偶尔会拿出来用一下，特别是疫情那会充当了 LD 的临时使用机器。<br>最近一次就是想把行车记录仪里的视频导出来，结果上传的时候我点了下 win10 的更新，重启后这个机器就起不来了，一开始以为顶多是个重装系统，结果重装也不行，就进不去，插着 U 盘也起不来，一开始还能进 BIOS，后面就直接进不去了，像内存条脏了，擦下金手指什么的，都没搞定，想着也只可能是主板上的部件坏了，所以就开始了这趟半作死之旅，买了块主板来换，拆这个笔记本是真的够难的，开机键那一条是真的拆不开，后面还把两个扣子扳坏了，里面的排线插口还几个也拔坏了，幸好用买来的主板装上还能用，但是后面就很奇怪了，因为这个电脑装了 Ubuntu 跟 win10 的双系统，Ubuntu 能正常起来，但是 win10 就是起不来，插上老毛桃或者 win10 的启动盘都是起不来，老毛桃是能起来，但是进不了 pe，插启动盘又报 <code>0xc00000e9</code> 错误码，现在想着是不是这个固态硬盘有点问题或者还是内存问题，只能后续再看了，拆机的时候也找不到视频，机器实在是太老了，后面再试试看。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Windows</category>
|
|
<category>小技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Windows</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>分享一次比较诡异的 Windows 下 U盘无法退出的经历</title>
|
|
<url>/2023/01/29/%E5%88%86%E4%BA%AB%E4%B8%80%E6%AC%A1%E6%AF%94%E8%BE%83%E8%AF%A1%E5%BC%82%E7%9A%84-Windows-%E4%B8%8B-U%E7%9B%98%E6%97%A0%E6%B3%95%E9%80%80%E5%87%BA%E7%9A%84%E7%BB%8F%E5%8E%86/</url>
|
|
<content><![CDATA[<p>作为一个 Windows 的老用户，并且也算是 Windows 系统的半个粉丝，但是秉承一贯的优缺点都应该说的原则，Windows 系统有一点缺点是真的挺难受，相信 Windows 用过比较久的都会经历过，就是 U盘无法退出的问题，在比较远古时代，这个问题似乎能采取的措施不多，关机再拔 U盘的方式是一种比较保险的方式，其他貌似有 360这种可以解除占用的，但是需要安装 360 软件，对于目前的使用环境来说有点得不偿失，也是比较流氓的一类软件了，目前在 Windows 环境我主要就安装了个火绒，或者就用 Windows 自带的 defender。</p>
|
|
<h2 id="第一种"><a href="#第一种" class="headerlink" title="第一种"></a>第一种</h2><p>最近这次经历也是有火绒的一定责任，在我尝试推出 U盘的时候提示了我被另一个大流氓软件，XlibabaProtect.exe 占用了，这个流氓软件真的是充分展示了某里的技术实力，试过 N 多种办法都关不掉也删不掉，尝试了很多种办法也没办法删除，但是后面换了种思路，一般这种情况肯定是有进程在占用 U盘里的内容，最新版本的 Powertoys 会在文件的右键菜单里添加一个叫 <a href="https://learn.microsoft.com/zh-cn/windows/powertoys/file-locksmith">File Locksmith</a> 的功能，可以用于检查正在使用哪些文件以及由哪些进程使用，但是可能是我的使用姿势不对，没有仔细看文档，它里面有个”以管理员身份重启”,可能会有用。<br>这算是第一种方式，</p>
|
|
<h2 id="第二种"><a href="#第二种" class="headerlink" title="第二种"></a>第二种</h2><p>第二种方式是 Windows 任务管理器中性能 tab 下的”打开资源监视器”，<br><img data-src="https://img.nicksxs.me/blog/or7HQY.png">,假如我的 U 盘的盘符是<code>F:</code><br><img data-src="https://img.nicksxs.me/blog/JYQ7ay.png">就可以搜索到占用这个盘符下文件的进程，这里千万小心‼️‼️，不可轻易杀掉这些进程，有些系统进程如果轻易杀掉会导致蓝屏等问题，不可轻易尝试，除非能确认这些进程的作用。<br>对于前两种方式对我来说都无效，</p>
|
|
<h2 id="第三种"><a href="#第三种" class="headerlink" title="第三种"></a>第三种</h2><p>所以尝试了第三种，<br>就是磁盘脱机的方式，在”计算机”右键管理，点击”磁盘管理”,可以找到 U 盘盘符右键，点击”脱机”，然后再”推出”，这个对我来说也不行</p>
|
|
<h2 id="第四种"><a href="#第四种" class="headerlink" title="第四种"></a>第四种</h2><p>这种是唯一对我有效的，在开始菜单搜索”event”，可以搜到”事件查看器”,<br><img data-src="https://img.nicksxs.me/blog/VmDVtY.png">，这个可以看到当前最近 Windows 发生的事件，打开这个后就点击U盘推出，因为推不出来也是一种错误事件，点击下刷新就能在这看到具体是因为什么推出不了，具体的进程信息<br>最后发现是英特尔的驱动管理程序的一个进程，关掉就退出了，虽然前面说的某里的进程是流氓，但这边是真的冤枉它了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Windows</category>
|
|
<category>小技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Windows</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>分享记录一下一个 git 操作方法</title>
|
|
<url>/2022/02/06/%E5%88%86%E4%BA%AB%E8%AE%B0%E5%BD%95%E4%B8%80%E4%B8%8B%E4%B8%80%E4%B8%AA-git-%E6%93%8D%E4%BD%9C%E6%96%B9%E6%B3%95/</url>
|
|
<content><![CDATA[<p>前阵子一个同事因为发现某个分支上的代码好像有缺失导致无法正常运行，然后就对比了下把缺失的代码从另一个分支上拷了过来，可能有所欠考虑，不过主要是说下操作过程和最后的处理方法，这位同学的操作是改一些代码commit 一下，这样的 commit 了大概五六次，并且已经 push 到了远端，然后就在想要怎么去处理，在本地可以 reset，已经到远端了，一个很不优雅的操作就是本地 reset 了用 force push，这个当然是不可取的，然后就是 revert 了，但是又已经 commit 了好几次了，网上看了下，好像处理方法还挺成熟的，git revert 命令本质上就是一个逆向的 git cherry-pick 操作。 它将你提交中的变更的以完全相反的方式的应用到一个新创建的提交中，本质上就是撤销或者倒转。可以理解为就是提交一个反向的操作，这里其实我们可以用<code>range revert</code>来进行 <code>git revert</code>, 用法就是</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">git revert OLDER_COMMIT^..NEWER_COMMIT</span><br></pre></td></tr></table></figure>
|
|
<p>这样就可以解决上面的问题了，但是还有个问题是这样会根据前面的 commit 数量提交对应数量个 revert commit 会显得比较乱，如果要比较干净的 commit 历史，<br>可以看下 <code>git revert</code> 命令说明<br><img data-src="https://img.nicksxs.me/uPic/3nmnwY.png"><br>然后就可以用 <code>-n</code> 参数，表示不自动提交</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">git revert -n OLDER_COMMIT^..NEWER_COMMIT</span><br><span class="line">git commit -m "revert OLDER_COMMIT to NEWER_COMMIT"</span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
]]></content>
|
|
<categories>
|
|
<category>git</category>
|
|
<category>小技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>git</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>分享记录一下一个 scp 操作方法</title>
|
|
<url>/2022/02/06/%E5%88%86%E4%BA%AB%E8%AE%B0%E5%BD%95%E4%B8%80%E4%B8%8B%E4%B8%80%E4%B8%AA-scp-%E6%93%8D%E4%BD%9C%E6%96%B9%E6%B3%95/</url>
|
|
<content><![CDATA[<p>scp 是个在服务器之间拷贝文件的一个常用命令，有时候有个场景是比如我们需要拷贝一些带有共同前缀的文件，但是有一个问题是比如我们有使用 zsh 的话，会出现一个报错，</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">zsh: no matches found: root@100.100.100.100://root/prefix*</span><br></pre></td></tr></table></figure>
|
|
<p>这里就比较奇怪了，这个前缀的文件肯定是有的，这里其实是由于 zsh 会对 <code>*</code> 进行展开，这个可以在例如 <code>ls</code> 命令在使用中就可以发现 <code>zsh</code> 有这个特性<br>需要使用双引号或单引号将路径包起来或者在<code>*</code>之前加反斜杠<code>\</code>来阻止对<code>*</code>展开和转义</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">scp root@100.100.100.100://root/prefix* .</span><br></pre></td></tr></table></figure>
|
|
<p>通过使用双引号<code>"</code>进行转义</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">scp root@100.100.100.100:"//root/prefix*" .</span><br></pre></td></tr></table></figure>
|
|
<p>或者可以将 shell 从 zsh 切换成 bash</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>shell</category>
|
|
<category>小技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>scp</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>使用milvus和towhee实现简陋版的以图搜图</title>
|
|
<url>/2024/08/04/%E4%BD%BF%E7%94%A8milvus%E5%92%8Ctowhee%E5%AE%9E%E7%8E%B0%E7%AE%80%E9%99%8B%E7%89%88%E7%9A%84%E4%BB%A5%E5%9B%BE%E6%90%9C%E5%9B%BE/</url>
|
|
<content><![CDATA[<p>上次我们尝试用 <code>towhee</code> 跟 <code>milvus</code> 实现了图片的向量化，那么顺势我们就能在这个基础上实现以图搜图<br>首先我们找一些图片，<br><img data-src="https://img.nicksxs.me/blog/mRuiAu.png"><br>然后我们先把他们都向量化，存储在milvus里</p>
|
|
<figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">from</span> towhee <span class="keyword">import</span> AutoPipes, AutoConfig, ops</span><br><span class="line"><span class="keyword">import</span> towhee</span><br><span class="line"><span class="keyword">import</span> os</span><br><span class="line"><span class="keyword">from</span> pymilvus <span class="keyword">import</span> MilvusClient</span><br><span class="line"><span class="keyword">import</span> json</span><br><span class="line"> </span><br><span class="line"><span class="comment"># 1. 设置一个Milvus客户端</span></span><br><span class="line">client = MilvusClient(</span><br><span class="line"> uri=<span class="string">"http://localhost:19530"</span></span><br><span class="line">)</span><br><span class="line">insert_conf = AutoConfig.load_config(<span class="string">'insert_milvus'</span>)</span><br><span class="line">insert_conf.collection_name = <span class="string">'text_image_search'</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line">insert_pipe = AutoPipes.pipeline(<span class="string">'insert_milvus'</span>, insert_conf)</span><br><span class="line"></span><br><span class="line"><span class="comment"># 创建图像嵌入管道</span></span><br><span class="line">image_embedding_pipe = AutoPipes.pipeline(<span class="string">'image-embedding'</span>)</span><br><span class="line">files = os.listdir(<span class="string">"./images"</span>)</span><br><span class="line"><span class="keyword">for</span> file <span class="keyword">in</span> files: </span><br><span class="line"> file_path = os.path.join(<span class="string">"./images"</span>, file)</span><br><span class="line"> <span class="keyword">if</span> os.path.isfile(file_path):</span><br><span class="line"> embedding = image_embedding_pipe(file_path).get()[<span class="number">0</span>]</span><br><span class="line"> insert_pipe([file_path, embedding])</span><br></pre></td></tr></table></figure>
|
|
<p>然后我们找一张神仙姐姐的其他图片，先把它 <code>embedding</code> 后在 <code>milvus</code> 里进行向量检索</p>
|
|
<figure class="highlight python"><table><tr><td class="code"><pre><span class="line">image_embedding_pipe = AutoPipes.pipeline(<span class="string">'image-embedding'</span>)</span><br><span class="line"></span><br><span class="line"><span class="comment"># 生成嵌入</span></span><br><span class="line">embedding = image_embedding_pipe(<span class="string">'./to_search.jpg'</span>).get()[<span class="number">0</span>]</span><br><span class="line"><span class="built_in">print</span>(embedding)</span><br><span class="line">res = client.search(</span><br><span class="line"> collection_name=<span class="string">"text_image_search"</span>,</span><br><span class="line"> data=[embedding],</span><br><span class="line"> limit=<span class="number">5</span>,</span><br><span class="line"> search_params={<span class="string">"metric_type"</span>: <span class="string">"IP"</span>, <span class="string">"params"</span>: {}}</span><br><span class="line">)</span><br><span class="line">result = json.dumps(res, indent=<span class="number">4</span>)</span><br><span class="line"><span class="built_in">print</span>(result)</span><br></pre></td></tr></table></figure>
|
|
<p>我们检索出来5个结果，<br><img data-src="https://img.nicksxs.me/blog/dKff2B.png"><br>可以通过distance找到距离最近的这个是id=451291280409722600<br>可以发现也是神仙姐姐的,只是作为参考<br>to_search 目标图片是<br><img data-src="https://img.nicksxs.me/blog/dm0YAx.png"><br>搜索的最短距离就是id对应的图片<br><img data-src="https://img.nicksxs.me/blog/F9q692.png"><br>这张图片就是神仙姐姐<br><img data-src="https://img.nicksxs.me/blog/ObdrMY.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>算法</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>算法</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>向量数据库 Milvus 安装初体验</title>
|
|
<url>/2024/07/21/%E5%90%91%E9%87%8F%E6%95%B0%E6%8D%AE%E5%BA%93-Milvus-%E5%AE%89%E8%A3%85%E5%88%9D%E4%BD%93%E9%AA%8C/</url>
|
|
<content><![CDATA[<p>之前做了个简单的铺垫，作为大模型应用技术领域非常重要的一环，向量数据库我们在前面有做一些引导性的介绍，其中的索引技术，<br>而在众多向量数据库比较有代表性的 Milvus，这边我们来尝试安装 Milvus 初体验一下<br>因为只是初体验，不作为生产环境使用，所以就用最简单的方式，Docker单机部署的方式<br>首先需要需要有Docker环境<br>确认下 docker 命令可执行<br>然后拉取启动脚本</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">wget https://raw.githubusercontent.com/milvus-io/milvus/master/scripts/standalone_embed.sh</span><br><span class="line">bash standalone_embed.sh start</span><br></pre></td></tr></table></figure>
|
|
<p>启动过程中需要注意一些问题，这也是有点想吐槽的，这个官方的启动脚本竟然都不是稳定成功的，最开始就是失败的，后面重新删除拉下来的镜像再启动就好了<br>并且官方提供的demo python版本的示例也是有问题的<br>因为主要是java的缘故，就用了java的sdk来尝试<br>只是我们可以先运行一个ui工具</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">docker run -p 8000:3000 -e MILVUS_URL={milvus server IP}:19530 zilliz/attu:v2.4</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/blog/fdca1a5d6bbe219358546866e933b554.png"><br>这里就能看到运行情况<br>然后我们运行个简单的代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">package</span> MilvusDemo;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> com.google.gson.Gson;</span><br><span class="line"><span class="keyword">import</span> com.google.gson.JsonObject;</span><br><span class="line"><span class="keyword">import</span> com.google.gson.JsonParser;</span><br><span class="line"><span class="keyword">import</span> io.milvus.v2.client.ConnectConfig;</span><br><span class="line"><span class="keyword">import</span> io.milvus.v2.client.MilvusClientV2;</span><br><span class="line"><span class="keyword">import</span> io.milvus.v2.common.DataType;</span><br><span class="line"><span class="keyword">import</span> io.milvus.v2.common.IndexParam;</span><br><span class="line"><span class="keyword">import</span> io.milvus.v2.service.collection.request.AddFieldReq;</span><br><span class="line"><span class="keyword">import</span> io.milvus.v2.service.collection.request.CreateCollectionReq;</span><br><span class="line"><span class="keyword">import</span> io.milvus.v2.service.index.request.CreateIndexReq;</span><br><span class="line"><span class="keyword">import</span> io.milvus.v2.service.vector.request.InsertReq;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> java.util.ArrayList;</span><br><span class="line"><span class="keyword">import</span> java.util.List;</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@author</span> shixuesen</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@date</span> 2024/7/21</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">Demo</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> <span class="type">ConnectConfig</span> <span class="variable">connectConfig</span> <span class="operator">=</span> ConnectConfig.builder()</span><br><span class="line"> .uri(<span class="string">"http://127.0.0.1:19530"</span>)</span><br><span class="line"> .build();</span><br><span class="line"> <span class="comment">// 连接配置</span></span><br><span class="line"> <span class="type">MilvusClientV2</span> <span class="variable">clientV2</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">MilvusClientV2</span>(connectConfig);</span><br><span class="line"> <span class="comment">// define a Collection Schema</span></span><br><span class="line"> CreateCollectionReq.<span class="type">CollectionSchema</span> <span class="variable">collectionSchema</span> <span class="operator">=</span> clientV2.createSchema();</span><br><span class="line"><span class="comment">// add two fileds, id and vector</span></span><br><span class="line"> <span class="type">Integer</span> <span class="variable">dim</span> <span class="operator">=</span> <span class="number">5</span>;</span><br><span class="line"> <span class="comment">// 这里的 CollectionSchema 就类似于mysql的数据库表结构</span></span><br><span class="line"> collectionSchema.addField(AddFieldReq.builder().fieldName(<span class="string">"my_id"</span>).dataType(DataType.Int64).isPrimaryKey(Boolean.TRUE).autoID(Boolean.FALSE).description(<span class="string">"id"</span>).build());</span><br><span class="line"> collectionSchema.addField(AddFieldReq.builder().fieldName(<span class="string">"my_vector"</span>).dataType(DataType.FloatVector).dimension(dim).build());</span><br><span class="line"></span><br><span class="line"> <span class="type">CreateCollectionReq</span> <span class="variable">req</span> <span class="operator">=</span> CreateCollectionReq</span><br><span class="line"> .builder()</span><br><span class="line"> .collectionSchema(collectionSchema)</span><br><span class="line"> .collectionName(<span class="string">"quick_setup"</span>)</span><br><span class="line"> .dimension(dim).build();</span><br><span class="line"> <span class="comment">// 通过schema 生成 Collection，就类似于mysql中的表</span></span><br><span class="line"> clientV2.createCollection(req);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 然后可以对字段创建索引</span></span><br><span class="line"> <span class="type">IndexParam</span> <span class="variable">indexParam</span> <span class="operator">=</span> IndexParam.builder()</span><br><span class="line"> .fieldName(<span class="string">"my_id"</span>)</span><br><span class="line"> .build();</span><br><span class="line"> <span class="type">IndexParam</span> <span class="variable">vertorIndexParam</span> <span class="operator">=</span> IndexParam.builder()</span><br><span class="line"> .fieldName(<span class="string">"my_vector"</span>)</span><br><span class="line"> .indexType(IndexParam.IndexType.AUTOINDEX)</span><br><span class="line"> .metricType(IndexParam.MetricType.IP)</span><br><span class="line"> .build();</span><br><span class="line"> List<IndexParam> indexParamList = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"> indexParamList.add(indexParam);</span><br><span class="line"> indexParamList.add(vertorIndexParam);</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">// 接着创建索引</span></span><br><span class="line"> clientV2.createIndex(CreateIndexReq.builder()</span><br><span class="line"> .collectionName(<span class="string">"quick_setup"</span>)</span><br><span class="line"> .indexParams(indexParamList)</span><br><span class="line"> .build());</span><br><span class="line"> <span class="type">JsonObject</span> <span class="variable">jsonObject</span> <span class="operator">=</span> JsonParser.parseString(<span class="string">"{\"my_id\": 0, \"my_vector\": [0.3580376395471989, -0.6023495712049978, 0.18414012509913835, -0.26286205330961354, 0.9029438446296592], \"color\": \"pink_8682\"}"</span>).getAsJsonObject();</span><br><span class="line"> List<JsonObject> jsonData = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"> jsonData.add(jsonObject);</span><br><span class="line"> <span class="comment">// 然后插入数据</span></span><br><span class="line"> clientV2.insert(InsertReq.builder().collectionName(<span class="string">"quick_setup"</span>)</span><br><span class="line"> .data(jsonData).build());</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 插入之后我们想进行查询，就用向量进行搜索，不过要先加载Collection</span></span><br><span class="line"> clientV2.loadCollection(LoadCollectionReq.builder().collectionName(<span class="string">"quick_setup"</span>).build());</span><br><span class="line"> <span class="type">SearchResp</span> <span class="variable">searchResp</span> <span class="operator">=</span> clientV2.search(SearchReq.builder()</span><br><span class="line"> .collectionName(<span class="string">"quick_setup"</span>)</span><br><span class="line"> .data(Collections.singletonList(<span class="keyword">new</span> <span class="title class_">FloatVec</span>(<span class="keyword">new</span> <span class="title class_">float</span>[]{<span class="number">0.3580376395471989F</span>, -<span class="number">0.6023495712049978F</span>, <span class="number">0.18414012509913835F</span>, -<span class="number">0.26286205330961354F</span>, <span class="number">0.9029438446296592F</span>})))</span><br><span class="line"> .topK(<span class="number">10</span>)</span><br><span class="line"> .outputFields(Collections.singletonList(<span class="string">"*"</span>))</span><br><span class="line"> .build()</span><br><span class="line"> );</span><br><span class="line"> <span class="keyword">for</span> (List<SearchResp.SearchResult> searchResult : searchResp.getSearchResults()) {</span><br><span class="line"> System.out.println(searchResult);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>我们用同样的向量进行搜索</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">[SearchResp.SearchResult(entity={my_id=<span class="number">0</span>, $meta={<span class="string">"color"</span>:<span class="string">"pink_8682"</span>}, my_vector=[<span class="number">0.35803765</span>, -<span class="number">0.6023496</span>, <span class="number">0.18414013</span>, -<span class="number">0.26286206</span>, <span class="number">0.90294385</span>]}, score=<span class="number">1.4093276</span>, id=<span class="number">0</span>)]</span><br></pre></td></tr></table></figure>
|
|
<p>得出的score就是很高的，只是做了下尝试，官方的示例代码是少得可怜，也不全，很难想象是目前比较热门的向量数据库</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>算法</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>算法</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>向量数据库学习基础之跳表</title>
|
|
<url>/2024/06/16/%E5%90%91%E9%87%8F%E6%95%B0%E6%8D%AE%E5%BA%93%E5%AD%A6%E4%B9%A0%E5%9F%BA%E7%A1%80%E4%B9%8B%E8%B7%B3%E8%A1%A8/</url>
|
|
<content><![CDATA[<p>在学习向量数据库的时候，发现检索的算法 <code>HNSW</code> 是个比较重要且核心的概念，直接学习（面向小白）可能有一定的门槛，所以我们先通过常见算法的跳表来做个引入<br>跳表的前置基础就是最常见的序列结构之一的链表，链表的特点是添加插入元素效率非常高，查询检索则需要线性复杂度，比如Java中的hashmap，在jdk1.8以后就把<br>单纯链表的结构改成了链表和红黑树结合的方式，因为当链表比较长的时候，线性时间复杂度也是个比较慢的方法相比对数时间复杂度，而对于链表，也有直接在它基础<br>优化而来的跳表结构，跳表是在链表的基础上引入了分层的概念，通过投硬币概率来决定是否生成新层，先用比较通俗的话来讲一下我的理解<br>因为链表查找只能从前往后一个一个找（单向链表），那么有没有办法可以跳过一些，加速我的查找，那么引入了分层的结构，在更上层以更稀疏的数据链表来索引数据，<br>举个例子，我原来是一个 </p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">1 --> 2 --> 3 --> 4 --> 5 --> 6 --> 7 --> 8 --> 9 </span><br></pre></td></tr></table></figure>
|
|
<p>这样的链表，我要查找8个元素是否存在，那我需要查找八次才能找到<br>如果我把结构改成</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">1 --------------> 4 --------------> 7 ---------> null </span><br><span class="line">| | |</span><br><span class="line">1 --> 2 --> 3 --> 4 --> 5 --> 6 --> 7 --> 8 --> 9 --> null</span><br></pre></td></tr></table></figure>
|
|
<p>变成这样的结构，我现在第一层找最后一个比8小的元素，就是7，然后再到下一层查找，这样我的查找路径就是 <code>1 --> 4 --> 7 --> 8</code> 速度加快了一倍<br>直观来说这样是能够提升很多查询效率，但是具体怎么实现我们也来看一下<br>首先定义个简单的Node</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">Node</span> {</span><br><span class="line"> <span class="keyword">public</span> <span class="type">int</span> <span class="variable">data</span> <span class="operator">=</span> -<span class="number">1</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> Node[] forwards;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">Node</span><span class="params">(<span class="type">int</span> level)</span> {</span><br><span class="line"> forwards = <span class="keyword">new</span> <span class="title class_">Node</span>[level];</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>接下去是SkipList的主体结构</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">SkipList</span> {</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">MAX_LEVEL</span> <span class="operator">=</span> <span class="number">16</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="type">int</span> <span class="variable">currentLevel</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="type">Node</span> <span class="variable">head</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Node</span>(MAX_LEVEL);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="type">Random</span> <span class="variable">random</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Random</span>();</span><br></pre></td></tr></table></figure>
|
|
<p>包含最大层数，当前层数，头结点，跟随机值<br>生成随机层数参考了redis的zset实现方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="type">int</span> <span class="title function_">generateRandomLevel</span><span class="params">()</span> {</span><br><span class="line"> <span class="comment">// 参考redis sorted set 实现</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">level</span> <span class="operator">=</span> <span class="number">1</span>;</span><br><span class="line"> <span class="keyword">while</span> ((random.nextInt() & <span class="number">0xFFFF</span>) < (<span class="number">0.25</span> * <span class="number">0xFFFF</span>))</span><br><span class="line"> level += <span class="number">1</span>;</span><br><span class="line"> <span class="keyword">return</span> Math.min(level, MAX_LEVEL);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>接下去先看下</p>
|
|
<h2 id="插入元素"><a href="#插入元素" class="headerlink" title="插入元素"></a>插入元素</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">insert</span><span class="params">(<span class="type">int</span> value)</span> {</span><br><span class="line"> <span class="type">int</span> <span class="variable">level</span> <span class="operator">=</span> generateRandomLevel();</span><br><span class="line"> <span class="keyword">if</span> (level >= currentLevel) {</span><br><span class="line"> currentLevel = level;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">Node</span> <span class="variable">newNode</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Node</span>(level);</span><br><span class="line"> newNode.setData(value);</span><br><span class="line"> Node[] update = <span class="keyword">new</span> <span class="title class_">Node</span>[level];</span><br><span class="line"></span><br><span class="line"> Arrays.fill(update, head);</span><br><span class="line"></span><br><span class="line"> <span class="type">Node</span> <span class="variable">p</span> <span class="operator">=</span> head;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> level - <span class="number">1</span>; i >= <span class="number">0</span>; --i) {</span><br><span class="line"> <span class="comment">// 找到应该处在的位置，把前一元素记录为待更新</span></span><br><span class="line"> <span class="keyword">while</span> (p.forwards[i] != <span class="literal">null</span> && p.forwards[i].data < value) {</span><br><span class="line"> p = p.forwards[i];</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 对应这一层，更新新的位置p</span></span><br><span class="line"> update[i] = p;</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="comment">// 每一层都处理前后链接</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < level; i++) {</span><br><span class="line"> newNode.forwards[i] = update[i].forwards[i];</span><br><span class="line"> update[i].forwards[i] = newNode;</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>看下图片演示,类似于链表的插入，但是需要处理每一层的前后链接<br><img data-src="https://img.nicksxs.me/blog/%E9%93%BE%E8%A1%A8.png"></p>
|
|
<h2 id="查找元素"><a href="#查找元素" class="headerlink" title="查找元素"></a>查找元素</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Node <span class="title function_">search</span><span class="params">(<span class="type">int</span> value)</span> {</span><br><span class="line"> <span class="type">Node</span> <span class="variable">p</span> <span class="operator">=</span> head;</span><br><span class="line"> <span class="comment">// 逐层找到比目标元素小的最后一个元素</span></span><br><span class="line"> <span class="comment">// 然后再往下一层找</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> currentLevel - <span class="number">1</span>; i >= <span class="number">0</span> ; i--) {</span><br><span class="line"> <span class="keyword">while</span> (p.forwards[i] != <span class="literal">null</span> && p.forwards[i].data < value) {</span><br><span class="line"> p = p.forwards[i];</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 此时如果前一元素就是目标元素就返回该元素</span></span><br><span class="line"> <span class="keyword">if</span> (p.forwards[<span class="number">0</span>] != <span class="literal">null</span> && p.forwards[<span class="number">0</span>].data == value) {</span><br><span class="line"> <span class="keyword">return</span> p.forwards[<span class="number">0</span>];</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>也可以看下图里的演示，按红色的线寻找元素<br><img data-src="https://img.nicksxs.me/blog/UbOCGZ.png"></p>
|
|
<h2 id="删除元素"><a href="#删除元素" class="headerlink" title="删除元素"></a>删除元素</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">delete</span><span class="params">(<span class="type">int</span> value)</span> {</span><br><span class="line"> Node[] update = <span class="keyword">new</span> <span class="title class_">Node</span>[currentLevel];</span><br><span class="line"> <span class="type">Node</span> <span class="variable">p</span> <span class="operator">=</span> head;</span><br><span class="line"> <span class="comment">// 找到待删除元素的前一元素</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> currentLevel - <span class="number">1</span>; i >= <span class="number">0</span>; i--) {</span><br><span class="line"> <span class="keyword">while</span> (p.forwards[i] != <span class="literal">null</span> && p.forwards[i].data < value) {</span><br><span class="line"> p = p.forwards[i];</span><br><span class="line"> }</span><br><span class="line"> update[i] = p;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 判断如果该层有这个元素就更新链接</span></span><br><span class="line"> <span class="keyword">if</span> (p.forwards[<span class="number">0</span>] != <span class="literal">null</span> && p.forwards[<span class="number">0</span>].data == value) {</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> currentLevel - <span class="number">1</span>; i >= <span class="number">0</span>; i--) {</span><br><span class="line"> <span class="keyword">if</span> (update[i].forwards[i] != <span class="literal">null</span> && update[i].forwards[i].data == value) {</span><br><span class="line"> update[i].forwards[i] = update[i].forwards[i].forwards[i];</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>删除操作就类似于插入操作。了解了跳表原理以后其实<code>HNSW</code>就是个在空间层面的跳表</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>算法</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>算法</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>周末我在老丈人家打了天小工</title>
|
|
<url>/2020/08/16/%E5%91%A8%E6%9C%AB%E6%88%91%E5%9C%A8%E8%80%81%E4%B8%88%E4%BA%BA%E5%AE%B6%E6%89%93%E4%BA%86%E5%A4%A9%E5%B0%8F%E5%B7%A5/</url>
|
|
<content><![CDATA[<p>这周回家提前约好了要去老丈人家帮下忙，因为在翻修下老房子，活不是特别整的那种，所以大部分都是自己干，或者找个大工临时干几天（我们这那种比较专业的泥工匠叫做大工），像我这样去帮忙的，就是干点小工（把给大工帮忙的，干些偏体力活的叫做小工）的活。从大学毕业以后真的蛮少帮家里干活了，以前上学的时候放假还是帮家里淘个米，简单的扫地拖地啥的，当然刚高考完的时候，还去我爸厂里帮忙干了几天的活，实在是比较累，不过现在想着是觉得自己那时候比较牛，而不是特别排斥这个活，相对于现在的工作来说，导致了一系列的职业病，颈椎腰背都很僵硬，眼镜也不好，还有反流，像我爸那种活反而是脑力加体力的比较好的结合。<br>这一天的活前半部分主要是在清理厨房，瓷砖上的油污和墙上天花板上即将脱落的石灰或者白色涂料层，这种活特别是瓷砖上的油污，之前在自己家里也干活，还是比较熟悉的，不过前面主要是LD 在干，我主要是先搞墙上和天花板上的，干活还是很需要技巧的，如果直接去铲，那基本我会变成一个灰人，而且吸一鼻子灰，老丈人比较专业，先接上软管用水冲，一冲效果特别好，有些石灰涂料层直接就冲掉了，冲完之后先用带加长杆的刀片铲铲了一圈墙面，说实话因为老房子之前租出去了，所以墙面什么的被糟蹋的比较难看，一层一层的，不过这还算还好，后面主要是天花板上的，这可难倒我了，从小我爸妈是比较把我当小孩管着，爬上爬下的基本都是我爸搞定，但是到了老丈人家也只得硬着头皮上了，爬到跳（一种建筑工地用的架子）上，还有点晃，小心脏扑通扑通跳，而且带加长杆的铲子还是比较重的，铲一会手也有点累，不过坚持着铲完了，上面还是比较平整的，不过下来的时候又把我难住了🤦♂️，往下爬的时候有根杆子要跨过去，由于裤子比较紧，强行一把跨过去怕抽筋，所以以一个非常尴尬的姿势停留休息了一会，再跨了过去，幸好事后问 LD，他们都没看到，哈哈哈，然后就是帮忙一起搞瓷砖上的油污，这个太有经验了，不过老丈人更有意思，一会试试啤酒，一会用用沙子，后面在午饭前基本就弄的比较干净了，就坐着等吃饭了，下午午休了会，就继续干活了。<br>下午是我这次体验的重点了，因为要清理以前贴的墙纸，真的是个很麻烦的活，只能说贴墙纸的师傅活干得太好了，基本不可能整个撕下来，想用铲子一点点铲下来也不行，太轻了就只铲掉表面一层，太重了就把墙纸跟墙面的石灰啥的整个铲下来了，而且手又累又酸，后来想着是不是继续用水冲一下，对着一小面墙试验了下，效果还不错，但是又发现了个问题，那一面墙又有一块是后面糊上去的，铲掉外层的石灰后不平，然后就是最最重头的，也是让我后遗症持续到第二天的，要把那一块糊上去的水泥敲下来，毛估下大概是敲了80%左右，剩下的我的手已经不会用力了，因为那一块应该是要糊上去的始作俑者，就一块里面凹进去的，我拿着榔头敲到我手已经没法使劲了，而且大下午，感觉没五分钟，我的汗已经糊满脸，眼睛也睁不开，不然就流到眼睛里了，此处获得成就一：用榔头敲墙壁，也是个技术加体力的活，而且需要非常好的技巧，否则手马上就废了，敲下去的反作用力，没一会就不行了，然后是看着老丈人兄弟帮忙拆一个柜子，在我看来是个几天都搞不定的活，他轻轻松松在我敲墙的那会就搞定了，以前总觉得我干的活非常有技术含量，可是这个事情真的也是很有技巧啊，它是个把一间房间分隔开的柜子，从底到顶上，还带着门，我还在旁边帮忙撬一下脚踢，一根木条撬半天，唉，成就二：专业的人就是不一样。<br>最后就是成就三了：我之前沾沾自喜的跑了多少步，做了什么锻炼，其实都是渣渣，像这样干一天活，没经历过的，基本大半天就废了，反过来说，如果能经常去这么干一天活，跑步啥的都是渣渣，消耗的能量远远超过跑个十公里啥的。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>运动</category>
|
|
<category>跑步</category>
|
|
<category>干活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>运动</tag>
|
|
<tag>减肥</tag>
|
|
<tag>跑步</tag>
|
|
<tag>干活</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>关于Termux中的shell命令的历史记录问题</title>
|
|
<url>/2025/04/06/%E5%85%B3%E4%BA%8ETermux%E4%B8%AD%E7%9A%84shell%E5%91%BD%E4%BB%A4%E7%9A%84%E5%8E%86%E5%8F%B2%E8%AE%B0%E5%BD%95%E9%97%AE%E9%A2%98/</url>
|
|
<content><![CDATA[<p>之前发现Termux在安卓手机中是个比较厉害的神器，就相当于一个随身携带的小型服务器，伴随着现在手机性能的逐渐强大，有些手机可能已经比很多个人用的云服务器还要强大很多，只是有着散热和电量的限制，但是充当一下临时的使用还是很不错的<br>记得之前提过，在用rustdesk的时候，如果被连端是Mac，并且被连端进入锁屏了，可能需要ssh连上去执行个唤醒命令，正好最近有需求要偶尔从手机连上家里的Mac，从而引发的一个问题是比如我经常用Termux来执行ssh命令，那么理论上我直接从shell的history就可以找到执行过的ssh命令，也就不用经常记对应的ip啥的，但是试用了下发现不行，一开始搜了下以为是没有设置历史存储的大小，可能默认是0，那么需要通过配置来设置</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">HISTSIZE=1000</span><br><span class="line">HISTFILESIZE=2000</span><br></pre></td></tr></table></figure>
|
|
<p>比如设置1000个命令大小，只是在Termux里不是这个原因，而是在于shell去保存历史的时机，安卓手机在退出应用或者切换应用程序都没法记录这个执行历史，而是需要在Termux中手动地用<code>exit</code>命令退出，才会记录下history历史，<br>当然也有人在Termux的issue上提到了，能不能捕捉安卓的事件来执行这个exit，否则这个exit有点难记得，相对来说也不方便了很多，容易遗忘，只是这个<a href="https://github.com/termux/termux-packages/issues/303">issue</a>也被关掉了，看看后面会不会优化吧</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>小技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>小技巧</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>在 wsl 2 中开启 ssh 连接</title>
|
|
<url>/2023/04/23/%E5%9C%A8-wsl-2-%E4%B8%AD%E5%BC%80%E5%90%AF-ssh-%E8%BF%9E%E6%8E%A5/</url>
|
|
<content><![CDATA[<p>之前在 wsl 1 中开启 ssh 其实很方便，只要把 sshd 服务起来就好了，但是在 wsl 2 中就不太一样了，<br>我这边使用的是 wsl 2 中的 Ubuntu 20.04，直接启动 sshd 服务是没法让其他机器连接的，而且都没有 ifconfig 命令可以查看 ip<br>不过可以用直接用 <code>ip a</code>来查看，可以看到这个 ip 是<code>172</code>网段的，而在 wsl 1 中可以看到 ip 就是 win 的 ip，<br>所以需要做一些操作，首先要安装 openssl-server</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sudo apt update</span><br><span class="line">sudo apt install openssh-server</span><br></pre></td></tr></table></figure>
|
|
<p>另外如果需要提高安全性，可以wsl 中配置 hosts.allow</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sshd:192.168.xx.</span><br></pre></td></tr></table></figure>
|
|
<p>先定一个子网段，然后对于ssh的配置，可以做以下修改</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">Port 22</span><br><span class="line">PasswordAuthentication yes</span><br></pre></td></tr></table></figure>
|
|
<p>在进行重启</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sudo service ssh --full-restart</span><br></pre></td></tr></table></figure>
|
|
<p>配置以后发现上面的问题，没法远程登录，因为 wsl 2 是基于 hyper-v 虚拟机实现的，并且 ip 使用的是一个虚拟出来的子网 ip，所以需要在 Windows 这一层配置端口的转发，可以通过命令<code>netsh interface portproxy show v4tov4</code>看到<br><img data-src="https://img.nicksxs.me/blog/WechatIMG605697.png"><br>截图是我已经添加好了的，先把原来的删除，再进行添加</p>
|
|
<figure class="highlight powershell"><table><tr><td class="code"><pre><span class="line">netsh interface portproxy delete v4tov4 listenaddress=<span class="number">0.0</span>.<span class="number">0.0</span> listenport=<span class="number">22</span></span><br><span class="line">netsh interface portproxy add v4tov4 listenaddress=<span class="number">0.0</span>.<span class="number">0.0</span> listenport=<span class="number">22</span> connectaddress=<span class="number">172.19</span>.<span class="number">129.207</span> connectport=<span class="number">22</span></span><br></pre></td></tr></table></figure>
|
|
<p>也可以全量删除</p>
|
|
<figure class="highlight powershell"><table><tr><td class="code"><pre><span class="line">netsh int portproxy reset all</span><br></pre></td></tr></table></figure>
|
|
<p>但是这样也不能直接访问了，还需要开启防火墙</p>
|
|
<figure class="highlight powershell"><table><tr><td class="code"><pre><span class="line">netsh advfirewall firewall add rule name=<span class="string">"WSL SSH"</span> <span class="built_in">dir</span>=<span class="keyword">in</span> action=allow protocol=TCP localport=<span class="number">22</span></span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
|
|
]]></content>
|
|
<categories>
|
|
<category>wsl</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>wsl</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>在claude code中体验下GLM最新模型</title>
|
|
<url>/2025/10/18/%E5%9C%A8claude-code%E4%B8%AD%E4%BD%93%E9%AA%8C%E4%B8%8BGLM%E6%9C%80%E6%96%B0%E6%A8%A1%E5%9E%8B/</url>
|
|
<content><![CDATA[<p>在很久之前最开始体验国产模型的时候就体验了ChatGLM，那个时候属于国内基本没几家有出大语言模型，但是说实话体验效果的确比较一般<br>但是目前GLM也是国内比较头部的大语言模型了，特别是最新出的GLM4.6，看大佬的评测好像比Deepseek V3.2还好一些<br>我们就基于claude code来做一下体验，还是老规矩，用最简单的prompt让它实现个todo应用<br>前置工作我们先去 <a href="https://bigmodel.cn/">bigmodel</a> 申请下apikey<br>申请完了就在我们体验的项目目录下</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">export ANTHROPIC_BASE_URL=``https://open.bigmodel.cn/api/anthropic</span><br><span class="line">export ANTHROPIC_AUTH_TOKEN="刚才复制的 API Key"</span><br></pre></td></tr></table></figure>
|
|
<p>一开始运行还发现模型是不对的，用的是claude的模型<br>需要手动指定</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">export ANTHROPIC_DEFAULT_SONNET_MODEL=GLM-4.5</span><br></pre></td></tr></table></figure>
|
|
<p>这里为什么用4.5呢，因为没付费，不给用4.6<br>所以我们先测一下4.5，待会再看下4.6<br>4.5模型的测了好几次都是中途卡住<br>生成的效果也比较一般<br><img data-src="https://img.nicksxs.me/uPic/DeZ5tR.png"><br>接下来我充了值，再试下4.6模型，果然动用了钞能力就是不一样，一遍完成，并且样式也美化了很多<br><img data-src="https://img.nicksxs.me/uPic/U82L0m.png"><br>过程中的任务编排也是比较合理，并且逐条完成了，4.5感觉还是完成度不高，4.6给我的感觉可能跟deepseek r1接近，<br>没有特别明显的优势，可能需要更加复杂的任务才能测出来差别，总之国产模型还是有在进步的</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>在老丈人家的小工记三</title>
|
|
<url>/2020/09/13/%E5%9C%A8%E8%80%81%E4%B8%88%E4%BA%BA%E5%AE%B6%E7%9A%84%E5%B0%8F%E5%B7%A5%E8%AE%B0%E4%B8%89/</url>
|
|
<content><![CDATA[<h2 id="小工记三"><a href="#小工记三" class="headerlink" title="小工记三"></a>小工记三</h2><p>前面这两周周末也都去老丈人家帮忙了，上上周周六先是去了那个在装修的旧房子那，把三楼收拾了下，因为要搬进来住，来不及等二楼装修好，就要把三楼里的东西都整理干净，这个活感觉是比较 easy，原来是就准备把三楼当放东西仓储的地方了，我们乡下大部分三层楼都是这么用的，这次也是没办法，之前搬进来的木头什么的都搬出去，主要是这上面灰尘太多，后面清理鼻孔的时候都是黑色的了，把东西都搬出去以后主要是地还是很脏，就扫了地拖了地，因为是水泥地，灰尘又太多了，拖起来都是会灰尘扬起来，整个脱完了的确干净很多，然而这会就出了个大乌龙，我们清理的是三楼的西边一间，结果老丈人上来说要住东边那间的🤦♂️，不过其实西边的也得清理，因为还是要放被子什么的，不算是白费功夫，接着清理东边那间，之前这个房子做过群租房，里面有个高低铺的床，当时觉得可以用在放被子什么的就没扔，只是拆掉了放旁边，我们就把它擦干净了又装好，发现螺丝🔩少了几个，亘古不变的真理，拆了以后装要不就多几个要不就少几个，不是很牢靠，不过用来放放被子省得放地上总还是可以的，对了前面还做了个事情就是铺地毯，其实也不是地毯，就是类似于墙布雨篷布那种，别人不用了送给我们的，三楼水泥地也不会铺瓷砖地板了就放一下，干净好看点，不过大小不合适要裁一下，那把剪刀是真的太难用了，我手都要抽筋了，它就是刀口只有一小个点是能剪下来的，其他都是钝的，后来还是用刀片直接裁，铺好以后，真的感觉也不太一样了，焕然一新的感觉<br>差不多中午了就去吃饭了，之前两次是去了一家小饭店，还是还比较干净，但是店里菜不好吃，还死贵，这次去了一家小快餐店，口味好，便宜，味道是真的不错，带鱼跟黄鱼都好吃，一点都不腥，我对这类比较腥的鱼真的是很挑剔的，基本上除了家里做的很少吃外面的，那天抱着试试的态度吃了下，真的还不错，后来丈母娘说好像这家老板是给别人结婚喜事酒席当厨师的，怪不得做的好吃，其实本来是有一点小抗拒，怕不干净什么的，后来发现菜很好吃，而且可能是老丈人跟干活的师傅去吃的比较多，老板很客气，我们吃完饭，还给我们买了葡萄吃，不过这家店有一个槽点，就是饭比较不好吃，有时候会夹生，不过后面聊起来其实是这种小菜馆饭点的通病，烧的太早太多容易多出来浪费，烧的迟了不够吃，而且大的电饭锅比较不容易烧好。<br>下午前面还是在处理三楼的，窗户上各种钉子，实在是太多了，我后面在走廊上排了一排🤦♂️，有些是直接断了，有些是就撬了出来，感觉我在杭州租房也没有这样子各种钉钉子，挂下衣服什么的也不用这么多吧，比较不能理解，搞得到处都是钉子。那天我爸也去帮忙了，主要是在卫生间里做白缝，其实也是个技术活，印象中好像我小时候自己家里也做过这个事情，但是比较模糊了，后面我们三楼搞完了就去帮我爸了，前面是我老婆二爹在那先刷上白缝，这里叫白缝，有些考究的也叫美缝，就是瓷砖铺完之后的缝，如果不去弄的话，里面水泥的颜色就露出来了，而且容易渗水，所以就要用白水泥加胶水搅拌之后糊在缝上，但是也不是直接糊，先要把缝抠一抠，因为铺瓷砖的还不会仔细到每个缝里的水泥都是一样满，而且也需要一些空间糊上去，不然就太表面的一层很容易被水直接冲掉了，然后这次其实也不是用的白水泥，而是直接现成买来就已经配好的用来填缝的，兑水搅拌均匀就好了，后面就主要是我跟我爸在搞，那个时候真的觉得我实在是太胖了，蹲下去真的没一会就受不了了，膝盖什么的太难受了，后面我跪着刷，然后膝盖又疼，也是比较不容易，不过我爸动作很快，我中间跪累了休息一会，我爸就能搞一大片，后面其实我也没做多少（谦虚一下），总体来讲这次不是很累，就是蹲着跪着腿有点受不了，是应该好好减肥了。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>运动</category>
|
|
<category>跑步</category>
|
|
<category>干活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>小技巧</tag>
|
|
<tag>运动</tag>
|
|
<tag>减肥</tag>
|
|
<tag>跑步</tag>
|
|
<tag>干活</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>在老丈人家的小工记五</title>
|
|
<url>/2020/10/18/%E5%9C%A8%E8%80%81%E4%B8%88%E4%BA%BA%E5%AE%B6%E7%9A%84%E5%B0%8F%E5%B7%A5%E8%AE%B0%E4%BA%94/</url>
|
|
<content><![CDATA[<p>终于回忆起来了，年纪大了写这种东西真的要立马写，不然很容易想不起来，那天应该是 9 月 12 日，也就是上周六，因为我爸也去了，而且娘亲（丈母娘，LD 这么叫，我也就随了她这么叫，当然是背后，当面就叫妈）也在那，早上一到那二爹就给我爸指挥了活，要挖一条院子的出水道，自己想出来的词，因为觉得下水道是竖的，在那稍稍帮了一会会忙，然后我还是比较惯例的跟着 LD 还有娘亲去住的家里，主要是老丈人可能也不太想让我干太累的活，因为上次已经差不多把三楼都整理干净了，然后就是二楼了，二楼说实话我也帮不上什么忙，主要是衣服被子什么的，正好是有张以前小孩子睡过的那种摇篮床，看上去虽然有一些破损，整体还是不错的，所以打算拿过去，我就负责把它拆掉了，比较简单的是只要拧螺丝就行了，但是其实是用了好多好多工具才搞定的，一开始只要螺丝刀就行了，但是因为年代久了，后面的螺帽也有点锈住或者本身就会串着会一起动，所以拿来了个扳手，大部分的其实都被这两个工具给搞定了，但是后期大概还剩下四分之一的时候，有一颗完全锈住，并且螺纹跟之前那些都不一样，但是这个已经是最大的螺丝刀了，也没办法换个大的了，所以又去找来个一字的，因为十字的不是也可以用一字的拧嘛，结果可能是我买的工具箱里的一字螺丝刀太新了，口子那很锋利，直接把螺丝花纹给划掉了，大的小的都划掉，然后真的变成凹进去一个圆柱体了，然后就想能不能隔一层布去拧，然而因为的确是已经变成圆柱体了，布也不太给力，不放弃的我又去找来了个老虎钳，妄图把划掉的螺丝用老虎钳钳住，另一端用扳手拧开螺帽，但是这个螺丝跟螺帽真的是生锈的太严重了，外加上钳不太牢，完全是两边一起转，实在是没办法了，在征得同意之后，直接掰断了，火死了，一颗螺丝折腾得比我拆一张床还久，那天因为早上去的也比较晚了，然后就快吃午饭了，正好想着带一点东西过去，就把一些脸盆，泡脚桶啥的拿过去了，先是去吃了饭，还是在那家快餐店，菜的口味还是依然不错，就是人比较多，我爸旁边都是素菜，都没怎么吃远一点的荤菜，下次要早点去，把荤菜放我爸旁边😄（PS：他们家饭还是依然尴尬，需要等），吃完就开到在修的房子那把东西拿了出来，我爸已经动作很快的打了一小半的地沟了，说实话那玩意真的是很重，我之前把它从三楼拿下来，就觉得这个太重了，这次还要用起来，感觉我的手会分分钟废掉，不过一开始我还是跟着LD去了住的家里，惯例睡了午觉，那天睡得比较踏实，竟然睡了一个小时，醒了想了下，其实LD她们收拾也用不上我（没啥力气活），我还是去帮我爸他们，跟LD说了下就去了在修的老房子那，两位老爹在一起钻地，看着就很累，我连忙上去想换一会他们，因为刚好是钻到混凝土地线，特别难，力道不够就会滑开，用蛮力就是钻进去拔不出来，原理是因为本身浇的时候就是很紧实的，需要边钻边动，那家伙实在是太重了，真的是汗如雨下，基本是三个人轮流来，我是个添乱的，经常卡住，然后把地线，其实就是一条混凝土横梁，里面还有14跟18的钢筋，需要割断，这个割断也是很有技巧，钢筋本身在里面是受到挤压的，直接用切割的，到快断掉的时候就会崩一下，非常危险，还是老丈人比较有经验，要留一点点，然后直接用榔头敲断就好了，本来以为这个是最难的了，结果下面是一块非常大的青基石，而且也是石头跟石头挤一块，边上一点点打钻有点杯水车薪，后来是用那种螺旋的钻，钻四个洞，相对位置大概是个长方形，这样子把中间这个长方形钻出来就比较容易地能拿出来了，后面的也容易搞出来了，后面的其实难度不是特别大了，主要是地沟打好之后得看看高低是不是符合要求的，不能本来是往外排水的反而外面高，这个怎么看就又很有技巧了，一般在地上的只要侧着看一下就好了，考究点就用下水平尺，但是在地下的，不用水平尺，其实可以借助于地沟里正要放进去的水管，放点水进去，看水往哪流就行了，铺好水管后，就剩填埋的活了，不是太麻烦了，那天真的是累到了，打那个混凝土的时候我真的是把我整个人压上去了，不过也挺爽的，有点把平时无处发泄的蛮力发泄出去了。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>运动</category>
|
|
<category>跑步</category>
|
|
<category>干活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>小技巧</tag>
|
|
<tag>运动</tag>
|
|
<tag>减肥</tag>
|
|
<tag>跑步</tag>
|
|
<tag>干活</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>在老丈人家的小工记四</title>
|
|
<url>/2020/09/26/%E5%9C%A8%E8%80%81%E4%B8%88%E4%BA%BA%E5%AE%B6%E7%9A%84%E5%B0%8F%E5%B7%A5%E8%AE%B0%E5%9B%9B/</url>
|
|
<content><![CDATA[<h2 id="小工记四"><a href="#小工记四" class="headerlink" title="小工记四"></a>小工记四</h2><p>第四周去的时候让我们去了现在在住的房子里，去三楼整理东西了，蛮多的东西需要收拾整理，有些需要丢一下，以前往往是把不太要用的东西就放三楼了，但是后面就不会再去收拾整理，LD 跟丈母娘负责收拾，我不太知道哪些还要的，哪些不要了，而且本来也不擅长这种收拾🤦♂️，然后就变成她们收拾出来废纸箱，我负责拆掉，压平，这时候终于觉得体重还算是有点作用，总体来说这个事情我其实也不擅长，不擅长的主要是捆起来，可能我总是小题大做，因为纸箱大小不一，如果不做一下分类，然后把大的折小一些的话，直接绑起来，容易拎起来就散掉了，而且一些鞋盒子这种小件的纸盒会比较薄，冰箱这种大件的比较厚，厚的比较不容易变形，需要大力踩踏，而且扎的时候需要用体重压住捆实了之后那样子才是真的捆实的，不然待会又是松松垮垮容易滑出来散架，因为压住了捆好后，下来了之后箱子就会弹开了把绳子崩紧实，感觉又是掌握到生活小技巧了😃，我这里其实比较单调无聊，然后 LD 那可以说非常厉害了，一共理出来 11 把旧电扇，还有好多没用过全新的不锈钢脸盆大大小小的，感觉比店里在卖的还多，还有是有比较多小时候的东西，特别多小时候的衣服，其实这种对我来说最难了，可能需要读一下断舍离，蛮多东西都舍不得扔，但是其实是没啥用了，然后还占地方，这天应该算是比较轻松的一天了，上午主要是把收拾出来要的和不要的搬下楼，然后下午要去把纸板给卖掉。中午还是去小快餐店吃的，在住的家里理东西还有个好处就是中午吃完饭可以小憩一下，因为我个人是非常依赖午休的，不然下午完全没精神，而且心态也会比较烦躁，一方面是客观的的确比较疲惫，另一方面应该主观心理作用也有点影响，就像上班的时候也是觉得不午睡就会很难受，心理作用也有一点，不过总之能睡还是睡一会，真的没办法就心态好点，吃完午饭之后我们就推着小平板车去收废品的地方卖掉了上午我收拾捆起来的纸板，好像卖了一百多，都是直接过地磅了，不用一捆一捆地称，不过有个小插曲，那里另外一个大爷在倒他的三轮车的时候撞了我一下，还好车速慢，屁股上肉垫后，接下来就比较麻烦了，是LD 她们两姐妹从小到大的书，也要去卖掉，小平板车就载不下了，而且着实也不太好推，轮子不太平，导致推着很累，书有好多箱，本来是想去亲戚家借电动三轮车，因为不会开摩托的那种，摩托的那种 LD 邻居家就有，可是到了那发现那个也是很大，而且刹车是用脚踩的那种，俺爹不太放心，就说第二天周日他有空会帮忙去载了卖掉的，然后比较搞笑的来了，丈母娘看错了时间，以为已经快五点了，就让我们顺便在车里带点东西去在修的房子，放到那边三楼去，到了那还跟老丈人说已经这么迟了要赶紧去菜场买菜了，结果我们回来以后才发现看错了一个小时🤦♂️。<br>前面可以没提，前三周去的我们一般就周六去一天，然后周日因为要早点回杭州，而且可能想让我们周日能休息下，但是这周就因为周日的时候我爸要去帮忙载书，然后 LD 姐姐也会过来收拾东西，我们周日就又去整理收拾了，周日由于俺爹去的很早，我过去的时候书已经木有了，主要是去收拾东西了，把一些有用没用的继续整理，基本上三楼的就处理完毕了，舒了一大口气，毕竟让丈母娘一个人收拾实在是太累了，但是要扔掉的衣服比较棘手，附近知道的青蛙回收桶被推倒了，其他地方也不知道哪里有，我们就先载了一些东西去在修的房子那，然后去找青蛙桶，结果一个小区可以进，但是已经满了，另一个不让进，后来只能让 LD 姐姐带去她们小区扔了，塞了满满一车。因为要赶回杭州的车就没有等我爸一起回来，他还在那帮忙搞卫生间的墙缝。<br>虽然这两天不太热，活也不算很吃力，不过我这个体重和易出汗的体质，还是让短袖不知道湿透了多少次，灌了好多水和冰红茶（下午能提提神），回来周一早上称体重也比较喜人，差一点就达到阶段目标，可以想想去哪里吃之前想好的烤肉跟火锅了（估计吃完立马回到解放前）。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>运动</category>
|
|
<category>跑步</category>
|
|
<category>干活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>小技巧</tag>
|
|
<tag>运动</tag>
|
|
<tag>减肥</tag>
|
|
<tag>跑步</tag>
|
|
<tag>干活</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>千万别升级Chrome-您的扩展程序可能无法使用-教你怎么解决</title>
|
|
<url>/2025/07/20/%E5%8D%83%E4%B8%87%E5%88%AB%E5%8D%87%E7%BA%A7Chrome-%E6%82%A8%E7%9A%84%E6%89%A9%E5%B1%95%E7%A8%8B%E5%BA%8F%E5%8F%AF%E8%83%BD%E6%97%A0%E6%B3%95%E4%BD%BF%E7%94%A8/</url>
|
|
<content><![CDATA[<p>允许我小小地标题党一下，后半句是真实情况，最近重启电脑就自动升级了chrome，发现很多必须的扩展程序都不能用了，之前是可以通过设置开启，这次升级到 138.0.7204.158 版本就直接不允许开启了<br>这个很久之前chrome也做了提示，主要是扩展程序的manifest版本要从v2升级成v3，但是这个升级分两方面说，一方面是升级难度，可以参考这个核对<a href="https://developer.chrome.com/docs/extensions/develop/migrate/checklist?hl=zh-cn">文档</a>, 需要</p>
|
|
<ul>
|
|
<li>迁移到 Service Worker</li>
|
|
<li>更新 API 调用</li>
|
|
<li>替换屏蔽 Web 请求监听器</li>
|
|
<li>提高扩展程序的安全性<br>这些项的修改，修改升级难度不小，如果使用了很多api的话是比较麻烦的<br>另一方面是很多扩展都不是自己做的开发，开发者都不一定在维护，但是很多都是非常深度使用的<br>这边有个例子就是<code>Proxy SwitchyOmega 2</code> 平时会用来做代理切换，原本的仓库已经没人维护了，并且chrome扩展商店也没有更新，<br>这里介绍两种方法，<br>第一种是找到这个应用所在目录<br><img data-src="https://img.nicksxs.me/chrome_extension_id.png"><br>首先找到应用的id，然后需要到chrome的扩展目录<br>mac是在这里 <code>~/Library/Application Support/Google/Chrome/Default/Extensions</code><br>windows是在 <code>C:\Users\<用户名>\AppData\Local\Google\Chrome\User Data\Default\Extensions</code><br>找到以后就找上面扩展的id就是扩展的文件目录，把它拷出来以后，然后找到扩展目录里的 <code>manifest.json</code> 文件，把版本改成3<br>然后再通过扩展路径直接打开，比如这个扩展就是 <code>chrome-extension://padekgcemlokbadohgkifijomclgjgif/options.html</code><br>这个选项页，为啥要这么打开是因为这个扩展的配置比较麻烦，就是现在可以搜到替代应用，但是要重配一遍也是挺头疼的<br>第二种是有点釜底抽薪的<br>可以在chrome的flag配置中修改<br>网上说的可能已经就删掉了，我这个版本的目前还可以用这个配置项来修改<br><code>chrome://flags/#allow-legacy-mv2-extensions</code><br>可以通过开启这个来让manifest版本是2的扩展能够被启用，但是这个肯定也是个临时方案，大家该把配置导出备份的尽早备份，同时也尽早寻找替代的升级了manifest 3的扩展程序</li>
|
|
</ul>
|
|
]]></content>
|
|
<categories>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>技巧</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>基于iflow的kimi模型来体验代码生成</title>
|
|
<url>/2025/11/16/%E5%9F%BA%E4%BA%8Eiflow%E7%9A%84kimi%E6%A8%A1%E5%9E%8B%E6%9D%A5%E4%BD%93%E9%AA%8C%E4%BB%A3%E7%A0%81%E7%94%9F%E6%88%90/</url>
|
|
<content><![CDATA[<p>首先是生成plan<br><img data-src="https://img.nicksxs.me/uPic/TcFxPi.png"></p>
|
|
<p>完整生成完后还是无法运行，多次把问题提给kimi也没有解决</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Compiled with warnings.</span><br><span class="line"></span><br><span class="line">[eslint]</span><br><span class="line">src/components/TodoItem.tsx</span><br><span class="line"> Line 4:19: 'FiX' is defined but never used @typescript-eslint/no-unused-vars</span><br><span class="line"> Line 4:108: 'FiClock' is defined but never used @typescript-eslint/no-unused-vars</span><br><span class="line"> Line 213:7: 'ActionBar' is assigned a value but never used @typescript-eslint/no-unused-vars</span><br><span class="line"> Line 222:7: 'ActionButtons' is assigned a value but never used @typescript-eslint/no-unused-vars</span><br><span class="line"> Line 334:7: 'Timestamp' is assigned a value but never used @typescript-eslint/no-unused-vars</span><br><span class="line"></span><br><span class="line">Search for the keywords to learn more about each warning.</span><br><span class="line">To ignore, add // eslint-disable-next-line to the line before.</span><br><span class="line"></span><br><span class="line">WARNING in [eslint]</span><br><span class="line">src/components/TodoItem.tsx</span><br><span class="line"> Line 4:19: 'FiX' is defined but never used @typescript-eslint/no-unused-vars</span><br><span class="line"> Line 4:108: 'FiClock' is defined but never used @typescript-eslint/no-unused-vars</span><br><span class="line"> Line 213:7: 'ActionBar' is assigned a value but never used @typescript-eslint/no-unused-vars</span><br><span class="line"> Line 222:7: 'ActionButtons' is assigned a value but never used @typescript-eslint/no-unused-vars</span><br><span class="line"> Line 334:7: 'Timestamp' is assigned a value but never used @typescript-eslint/no-unused-vars</span><br><span class="line"></span><br><span class="line">webpack compiled with 1 warning</span><br><span class="line">Files successfully emitted, waiting for typecheck results...</span><br><span class="line">Issues checking in progress...</span><br><span class="line">ERROR in src/components/Header.tsx:270:16</span><br><span class="line">TS2786: 'FiSearch' cannot be used as a JSX component.</span><br><span class="line"> Its return type 'ReactNode' is not a valid JSX element.</span><br><span class="line"> Type 'undefined' is not assignable to type 'Element | null'.</span><br><span class="line"> 268 | <SearchContainer></span><br><span class="line"> 269 | <SearchIcon></span><br><span class="line"> > 270 | <FiSearch {...({ size: 16 } as any)} /></span><br><span class="line"> | ^^^^^^^^</span><br><span class="line"> 271 | </SearchIcon></span><br><span class="line"> 272 | <SearchInput</span><br><span class="line"> 273 | type="text"</span><br><span class="line"></span><br><span class="line">ERROR in src/components/Header.tsx:300:18</span><br><span class="line">TS2786: 'FiHome' cannot be used as a JSX component.</span><br><span class="line"> Its return type 'ReactNode' is not a valid JSX element.</span><br><span class="line"> 298 | <NavIcons></span><br><span class="line"> 299 | <NavIcon></span><br><span class="line"> > 300 | <FiHome {...({ size: 24 } as any)} /></span><br><span class="line"> | ^^^^^^</span><br><span class="line"> 301 | </NavIcon></span><br><span class="line"> 302 | <NavIcon></span><br><span class="line"> 303 | <FiCompass {...({ size: 24 } as any)} /></span><br><span class="line"></span><br><span class="line"></span><br><span class="line">ERROR in src/components/TodoItem.tsx:453:43</span><br><span class="line">TS2786: 'FiTag' cannot be used as a JSX component.</span><br><span class="line"> Its return type 'ReactNode' is not a valid JSX element.</span><br><span class="line"> 451 | {todo.priority === 'high' ? '🔴 高' : todo.priority === 'medium' ? '🟡 中' : '🟢 低'}</span><br><span class="line"> 452 | </PriorityBadge></span><br><span class="line"> > 453 | {todo.category && <CategoryTag><FiTag size={10} /> #{todo.category}</CategoryTag>}</span><br><span class="line"> | ^^^^^</span><br><span class="line"> 454 |</span><br><span class="line"> 455 | <div style={{ position: 'relative' }}></span><br><span class="line"> 456 | <MoreButton onClick={() => setShowDropdown(!showDropdown)}></span><br><span class="line"></span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
<p>没办法，后面我好好思考了下，前面因为响应一直很慢，我退出了几次又继续使用同一个prompt，但是实际的模型是需要读取原先写到一半的代码，这样其实每次的生成上下文就被搞混乱了，所以重新清理了代码，再给kimi一次机会，这次就慢慢等了<br>首先也是列出了todo list<br><img data-src="https://img.nicksxs.me/uPic/IzMoRN.png"><br>完成后主体的代码也不多</p>
|
|
<figure class="highlight tsx"><table><tr><td class="code"><pre><span class="line"><span class="keyword">import</span> <span class="title class_">React</span>, { useState, useEffect } <span class="keyword">from</span> <span class="string">'react'</span>;</span><br><span class="line"><span class="keyword">import</span> styled <span class="keyword">from</span> <span class="string">'@emotion/styled'</span>;</span><br><span class="line"><span class="keyword">import</span> { motion, <span class="title class_">AnimatePresence</span> } <span class="keyword">from</span> <span class="string">'framer-motion'</span>;</span><br><span class="line"><span class="keyword">import</span> { <span class="title class_">Plus</span> } <span class="keyword">from</span> <span class="string">'lucide-react'</span>;</span><br><span class="line"><span class="keyword">import</span> <span class="title class_">TodoItem</span> <span class="keyword">from</span> <span class="string">'./TodoItem'</span>;</span><br><span class="line"><span class="keyword">import</span> <span class="title class_">TodoForm</span> <span class="keyword">from</span> <span class="string">'./TodoForm'</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">export</span> <span class="keyword">interface</span> <span class="title class_">Todo</span> {</span><br><span class="line"> <span class="attr">id</span>: <span class="built_in">string</span>;</span><br><span class="line"> <span class="attr">text</span>: <span class="built_in">string</span>;</span><br><span class="line"> <span class="attr">completed</span>: <span class="built_in">boolean</span>;</span><br><span class="line"> <span class="attr">createdAt</span>: <span class="title class_">Date</span>;</span><br><span class="line"> <span class="attr">priority</span>: <span class="string">'low'</span> | <span class="string">'medium'</span> | <span class="string">'high'</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">Container</span> = <span class="title function_">styled</span>(motion.<span class="property">div</span>)<span class="string">`</span></span><br><span class="line"><span class="string"> background: rgba(255, 255, 255, 0.95);</span></span><br><span class="line"><span class="string"> backdrop-filter: blur(20px);</span></span><br><span class="line"><span class="string"> border-radius: 24px;</span></span><br><span class="line"><span class="string"> box-shadow: 0 25px 50px -12px rgba(0, 0, 0, 0.25);</span></span><br><span class="line"><span class="string"> width: 100%;</span></span><br><span class="line"><span class="string"> max-width: 500px;</span></span><br><span class="line"><span class="string"> min-height: 600px;</span></span><br><span class="line"><span class="string"> padding: 30px;</span></span><br><span class="line"><span class="string"> position: relative;</span></span><br><span class="line"><span class="string"> overflow: hidden;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">Header</span> = styled.<span class="property">div</span><span class="string">`</span></span><br><span class="line"><span class="string"> text-align: center;</span></span><br><span class="line"><span class="string"> margin-bottom: 30px;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">Title</span> = styled.<span class="property">h1</span><span class="string">`</span></span><br><span class="line"><span class="string"> font-size: 2.5rem;</span></span><br><span class="line"><span class="string"> font-weight: 700;</span></span><br><span class="line"><span class="string"> background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);</span></span><br><span class="line"><span class="string"> -webkit-background-clip: text;</span></span><br><span class="line"><span class="string"> -webkit-text-fill-color: transparent;</span></span><br><span class="line"><span class="string"> background-clip: text;</span></span><br><span class="line"><span class="string"> margin: 0;</span></span><br><span class="line"><span class="string"> letter-spacing: -0.02em;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">Subtitle</span> = styled.<span class="property">p</span><span class="string">`</span></span><br><span class="line"><span class="string"> color: #6b7280;</span></span><br><span class="line"><span class="string"> font-size: 1rem;</span></span><br><span class="line"><span class="string"> margin: 8px 0 0 0;</span></span><br><span class="line"><span class="string"> font-weight: 500;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">TodoList</span> = styled.<span class="property">div</span><span class="string">`</span></span><br><span class="line"><span class="string"> margin-top: 30px;</span></span><br><span class="line"><span class="string"> max-height: 400px;</span></span><br><span class="line"><span class="string"> overflow-y: auto;</span></span><br><span class="line"><span class="string"> padding-right: 10px;</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &::-webkit-scrollbar {</span></span><br><span class="line"><span class="string"> width: 6px;</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &::-webkit-scrollbar-track {</span></span><br><span class="line"><span class="string"> background: #f1f5f9;</span></span><br><span class="line"><span class="string"> border-radius: 3px;</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &::-webkit-scrollbar-thumb {</span></span><br><span class="line"><span class="string"> background: #cbd5e1;</span></span><br><span class="line"><span class="string"> border-radius: 3px;</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &::-webkit-scrollbar-thumb:hover {</span></span><br><span class="line"><span class="string"> background: #94a3b8;</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">EmptyState</span> = <span class="title function_">styled</span>(motion.<span class="property">div</span>)<span class="string">`</span></span><br><span class="line"><span class="string"> text-align: center;</span></span><br><span class="line"><span class="string"> padding: 60px 20px;</span></span><br><span class="line"><span class="string"> color: #9ca3af;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">EmptyIcon</span> = styled.<span class="property">div</span><span class="string">`</span></span><br><span class="line"><span class="string"> font-size: 4rem;</span></span><br><span class="line"><span class="string"> margin-bottom: 16px;</span></span><br><span class="line"><span class="string"> opacity: 0.5;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">EmptyText</span> = styled.<span class="property">p</span><span class="string">`</span></span><br><span class="line"><span class="string"> font-size: 1.1rem;</span></span><br><span class="line"><span class="string"> margin: 0;</span></span><br><span class="line"><span class="string"> font-weight: 500;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">Stats</span> = styled.<span class="property">div</span><span class="string">`</span></span><br><span class="line"><span class="string"> display: flex;</span></span><br><span class="line"><span class="string"> justify-content: space-between;</span></span><br><span class="line"><span class="string"> align-items: center;</span></span><br><span class="line"><span class="string"> margin-top: 20px;</span></span><br><span class="line"><span class="string"> padding-top: 20px;</span></span><br><span class="line"><span class="string"> border-top: 1px solid #e5e7eb;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">StatItem</span> = styled.<span class="property">div</span><span class="string">`</span></span><br><span class="line"><span class="string"> text-align: center;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">StatNumber</span> = styled.<span class="property">div</span><span class="string">`</span></span><br><span class="line"><span class="string"> font-size: 1.5rem;</span></span><br><span class="line"><span class="string"> font-weight: 700;</span></span><br><span class="line"><span class="string"> color: #667eea;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">StatLabel</span> = styled.<span class="property">div</span><span class="string">`</span></span><br><span class="line"><span class="string"> font-size: 0.875rem;</span></span><br><span class="line"><span class="string"> color: #6b7280;</span></span><br><span class="line"><span class="string"> margin-top: 4px;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">TodoApp</span>: <span class="title class_">React</span>.<span class="property">FC</span> = <span class="function">() =></span> {</span><br><span class="line"> <span class="keyword">const</span> [todos, setTodos] = useState<<span class="title class_">Todo</span>[]>([]);</span><br><span class="line"> <span class="keyword">const</span> [showForm, setShowForm] = <span class="title function_">useState</span>(<span class="literal">false</span>);</span><br><span class="line"> <span class="keyword">const</span> [editingTodo, setEditingTodo] = useState<<span class="title class_">Todo</span> | <span class="literal">null</span>>(<span class="literal">null</span>);</span><br><span class="line"></span><br><span class="line"> <span class="title function_">useEffect</span>(<span class="function">() =></span> {</span><br><span class="line"> <span class="keyword">const</span> savedTodos = <span class="variable language_">localStorage</span>.<span class="title function_">getItem</span>(<span class="string">'todos'</span>);</span><br><span class="line"> <span class="keyword">if</span> (savedTodos) {</span><br><span class="line"> <span class="title function_">setTodos</span>(<span class="title class_">JSON</span>.<span class="title function_">parse</span>(savedTodos).<span class="title function_">map</span>(<span class="function">(<span class="params">todo: <span class="built_in">any</span></span>) =></span> ({</span><br><span class="line"> ...todo,</span><br><span class="line"> <span class="attr">createdAt</span>: <span class="keyword">new</span> <span class="title class_">Date</span>(todo.<span class="property">createdAt</span>)</span><br><span class="line"> })));</span><br><span class="line"> }</span><br><span class="line"> }, []);</span><br><span class="line"></span><br><span class="line"> <span class="title function_">useEffect</span>(<span class="function">() =></span> {</span><br><span class="line"> <span class="variable language_">localStorage</span>.<span class="title function_">setItem</span>(<span class="string">'todos'</span>, <span class="title class_">JSON</span>.<span class="title function_">stringify</span>(todos));</span><br><span class="line"> }, [todos]);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">const</span> <span class="title function_">addTodo</span> = (<span class="params">text: <span class="built_in">string</span>, priority: <span class="string">'low'</span> | <span class="string">'medium'</span> | <span class="string">'high'</span></span>) => {</span><br><span class="line"> <span class="keyword">const</span> <span class="attr">newTodo</span>: <span class="title class_">Todo</span> = {</span><br><span class="line"> <span class="attr">id</span>: <span class="title class_">Date</span>.<span class="title function_">now</span>().<span class="title function_">toString</span>(),</span><br><span class="line"> text,</span><br><span class="line"> <span class="attr">completed</span>: <span class="literal">false</span>,</span><br><span class="line"> <span class="attr">createdAt</span>: <span class="keyword">new</span> <span class="title class_">Date</span>(),</span><br><span class="line"> priority</span><br><span class="line"> };</span><br><span class="line"> <span class="title function_">setTodos</span>([newTodo, ...todos]);</span><br><span class="line"> <span class="title function_">setShowForm</span>(<span class="literal">false</span>);</span><br><span class="line"> };</span><br><span class="line"></span><br><span class="line"> <span class="keyword">const</span> <span class="title function_">toggleTodo</span> = (<span class="params">id: <span class="built_in">string</span></span>) => {</span><br><span class="line"> <span class="title function_">setTodos</span>(todos.<span class="title function_">map</span>(<span class="function"><span class="params">todo</span> =></span></span><br><span class="line"> todo.<span class="property">id</span> === id ? { ...todo, <span class="attr">completed</span>: !todo.<span class="property">completed</span> } : todo</span><br><span class="line"> ));</span><br><span class="line"> };</span><br><span class="line"></span><br><span class="line"> <span class="keyword">const</span> <span class="title function_">deleteTodo</span> = (<span class="params">id: <span class="built_in">string</span></span>) => {</span><br><span class="line"> <span class="title function_">setTodos</span>(todos.<span class="title function_">filter</span>(<span class="function"><span class="params">todo</span> =></span> todo.<span class="property">id</span> !== id));</span><br><span class="line"> };</span><br><span class="line"></span><br><span class="line"> <span class="keyword">const</span> <span class="title function_">editTodo</span> = (<span class="params">id: <span class="built_in">string</span>, newText: <span class="built_in">string</span>, newPriority: <span class="string">'low'</span> | <span class="string">'medium'</span> | <span class="string">'high'</span></span>) => {</span><br><span class="line"> <span class="title function_">setTodos</span>(todos.<span class="title function_">map</span>(<span class="function"><span class="params">todo</span> =></span></span><br><span class="line"> todo.<span class="property">id</span> === id ? { ...todo, <span class="attr">text</span>: newText, <span class="attr">priority</span>: newPriority } : todo</span><br><span class="line"> ));</span><br><span class="line"> <span class="title function_">setEditingTodo</span>(<span class="literal">null</span>);</span><br><span class="line"> };</span><br><span class="line"></span><br><span class="line"> <span class="keyword">const</span> completedCount = todos.<span class="title function_">filter</span>(<span class="function"><span class="params">todo</span> =></span> todo.<span class="property">completed</span>).<span class="property">length</span>;</span><br><span class="line"> <span class="keyword">const</span> totalCount = todos.<span class="property">length</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> (</span><br><span class="line"> <span class="language-xml"><span class="tag"><<span class="name">Container</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">initial</span>=<span class="string">{{</span> <span class="attr">opacity:</span> <span class="attr">0</span>, <span class="attr">y:</span> <span class="attr">20</span> }}</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">animate</span>=<span class="string">{{</span> <span class="attr">opacity:</span> <span class="attr">1</span>, <span class="attr">y:</span> <span class="attr">0</span> }}</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">transition</span>=<span class="string">{{</span> <span class="attr">duration:</span> <span class="attr">0.6</span>, <span class="attr">ease:</span> "<span class="attr">easeOut</span>" }}</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> ></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">Header</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">Title</span>></span>My Tasks<span class="tag"></<span class="name">Title</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">Subtitle</span>></span>Stay organized and productive<span class="tag"></<span class="name">Subtitle</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">Header</span>></span></span></span><br><span class="line"><span class="language-xml"></span></span><br><span class="line"><span class="language-xml"> {showForm && (</span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">TodoForm</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">onSubmit</span>=<span class="string">{addTodo}</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">onCancel</span>=<span class="string">{()</span> =></span> setShowForm(false)}</span></span><br><span class="line"><span class="language-xml"> /></span></span><br><span class="line"><span class="language-xml"> )}</span></span><br><span class="line"><span class="language-xml"></span></span><br><span class="line"><span class="language-xml"> {editingTodo && (</span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">TodoForm</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">todo</span>=<span class="string">{editingTodo}</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">onSubmit</span>=<span class="string">{(text,</span> <span class="attr">priority</span>) =></span> editTodo(editingTodo.id, text, priority)}</span></span><br><span class="line"><span class="language-xml"> onCancel={() => setEditingTodo(null)}</span></span><br><span class="line"><span class="language-xml"> /></span></span><br><span class="line"><span class="language-xml"> )}</span></span><br><span class="line"><span class="language-xml"></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">TodoList</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">AnimatePresence</span>></span></span></span><br><span class="line"><span class="language-xml"> {todos.length === 0 ? (</span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">EmptyState</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">initial</span>=<span class="string">{{</span> <span class="attr">opacity:</span> <span class="attr">0</span> }}</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">animate</span>=<span class="string">{{</span> <span class="attr">opacity:</span> <span class="attr">1</span> }}</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">exit</span>=<span class="string">{{</span> <span class="attr">opacity:</span> <span class="attr">0</span> }}</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> ></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">EmptyIcon</span>></span>📝<span class="tag"></<span class="name">EmptyIcon</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">EmptyText</span>></span>No tasks yet. Add one to get started!<span class="tag"></<span class="name">EmptyText</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">EmptyState</span>></span></span></span><br><span class="line"><span class="language-xml"> ) : (</span></span><br><span class="line"><span class="language-xml"> todos.map((todo) => (</span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">TodoItem</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">key</span>=<span class="string">{todo.id}</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">todo</span>=<span class="string">{todo}</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">onToggle</span>=<span class="string">{toggleTodo}</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">onDelete</span>=<span class="string">{deleteTodo}</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">onEdit</span>=<span class="string">{setEditingTodo}</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> /></span></span></span><br><span class="line"><span class="language-xml"> ))</span></span><br><span class="line"><span class="language-xml"> )}</span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">AnimatePresence</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">TodoList</span>></span></span></span><br><span class="line"><span class="language-xml"></span></span><br><span class="line"><span class="language-xml"> {todos.length > 0 && (</span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">Stats</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatItem</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatNumber</span>></span>{totalCount}<span class="tag"></<span class="name">StatNumber</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatLabel</span>></span>Total<span class="tag"></<span class="name">StatLabel</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">StatItem</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatItem</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatNumber</span>></span>{completedCount}<span class="tag"></<span class="name">StatNumber</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatLabel</span>></span>Completed<span class="tag"></<span class="name">StatLabel</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">StatItem</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatItem</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatNumber</span>></span>{totalCount - completedCount}<span class="tag"></<span class="name">StatNumber</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatLabel</span>></span>Remaining<span class="tag"></<span class="name">StatLabel</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">StatItem</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">Stats</span>></span></span></span><br><span class="line"><span class="language-xml"> )}</span></span><br><span class="line"><span class="language-xml"></span></span><br><span class="line"><span class="language-xml"> {!showForm && !editingTodo && (</span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">motion.button</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">style</span>=<span class="string">{{</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">position:</span> '<span class="attr">absolute</span>',</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">bottom:</span> '<span class="attr">30px</span>',</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">right:</span> '<span class="attr">30px</span>',</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">width:</span> '<span class="attr">60px</span>',</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">height:</span> '<span class="attr">60px</span>',</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">borderRadius:</span> '<span class="attr">50</span>%',</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">background:</span> '<span class="attr">linear-gradient</span>(<span class="attr">135deg</span>, #<span class="attr">667eea</span> <span class="attr">0</span>%, #<span class="attr">764ba2</span> <span class="attr">100</span>%)',</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">border:</span> '<span class="attr">none</span>',</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">color:</span> '<span class="attr">white</span>',</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">fontSize:</span> '<span class="attr">24px</span>',</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">cursor:</span> '<span class="attr">pointer</span>',</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">boxShadow:</span> '<span class="attr">0</span> <span class="attr">10px</span> <span class="attr">25px</span> <span class="attr">rgba</span>(<span class="attr">102</span>, <span class="attr">126</span>, <span class="attr">234</span>, <span class="attr">0.4</span>)',</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">display:</span> '<span class="attr">flex</span>',</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">alignItems:</span> '<span class="attr">center</span>',</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">justifyContent:</span> '<span class="attr">center</span>'</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> }}</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">whileHover</span>=<span class="string">{{</span> <span class="attr">scale:</span> <span class="attr">1.1</span> }}</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">whileTap</span>=<span class="string">{{</span> <span class="attr">scale:</span> <span class="attr">0.95</span> }}</span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">onClick</span>=<span class="string">{()</span> =></span> setShowForm(true)}</span></span><br><span class="line"><span class="language-xml"> ></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">Plus</span> <span class="attr">size</span>=<span class="string">{24}</span> /></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">motion.button</span>></span></span></span><br><span class="line"><span class="language-xml"> )}</span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">Container</span>></span></span></span><br><span class="line"> );</span><br><span class="line">};</span><br><span class="line"></span><br><span class="line"><span class="keyword">export</span> <span class="keyword">default</span> <span class="title class_">TodoApp</span>;</span><br></pre></td></tr></table></figure>
|
|
<p>一开始运行出现了个问题，<br><code>新增的时候报错 Component selectors can only be used in conjunction with @emotion/babel-plugin, the swc Emotion plugin, or another Emotion-aware compiler transform.</code><br>就继续让kimi来解决这个问题，结果一次就改好了，<br><img data-src="https://img.nicksxs.me/uPic/eZeAMi.png"><br>整体看起来也还行，可能的确是我一开始中断了几次导致上下文太复杂，只是不确定是模型消耗资源比较多还是什么原因响应有时候会很慢，可能免费的资源池是共用的，高峰期就比较慢了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>关于arthas的一个比较有用的使用方式</title>
|
|
<url>/2025/04/13/%E5%85%B3%E4%BA%8Earthas%E7%9A%84%E4%B8%80%E4%B8%AA%E6%AF%94%E8%BE%83%E6%9C%89%E7%94%A8%E7%9A%84%E4%BD%BF%E7%94%A8%E6%96%B9%E5%BC%8F/</url>
|
|
<content><![CDATA[<p>arthas是阿里开源的一个非常好用的java诊断工具，提供了很多很好用的命令，这里讲一个最近使用到的<br>就是将arthas挂载上我们的springboot应用，然后调用其中的方法，这样能够在如果没加日志已经看不到函数返回时更方便的排查问题<br>首先举个例子，我们有个Controller<br>它的一个query方法是这样的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@RequestMapping(value = "/query", method = RequestMethod.GET)</span></span><br><span class="line"><span class="meta">@ResponseBody</span></span><br><span class="line"><span class="keyword">public</span> String <span class="title function_">query</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> demoService.queryName(<span class="string">"1"</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>而在这个demoService中它的实现是这样</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> String <span class="title function_">queryName</span><span class="params">(String no)</span> {</span><br><span class="line"> <span class="keyword">if</span> (<span class="string">"1"</span>.equals(no)) {</span><br><span class="line"> <span class="keyword">return</span> <span class="string">"no1"</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="string">"no2"</span>;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>假如现在Controller这的这个方法有点问题，那么我想确认下是不是demoService这个方法的实现有问题，或者说确定下它的返回值是否符合预期<br>那么我们就可以在应用启动后，运行arthas，找到这个应用的进程，进行挂载<br>然后执行</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">vmtool --action getInstances --className com.nicksxs.spbdemo.service.DemoServiceImpl --express 'instances[0].queryName("1")'</span><br></pre></td></tr></table></figure>
|
|
<p>先介绍下这个<code>vmtool</code>命令<br>主要来说 vmtool 可以利用JVMTI接口，实现查询内存对象，强制 GC 等功能。<br>例如官方示例里的，我想把内存里的string对象捞一些出来看看存的是啥</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">vmtool --action getInstances --className java.lang.String --limit 10</span><br></pre></td></tr></table></figure>
|
|
<p>就可以这样，首先这个action就是指定要做的操作，支持的action 还包括</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">forceGc</span><br><span class="line">interruptThread</span><br></pre></td></tr></table></figure>
|
|
<p>等，那么对于 <code>getInstances</code> 就是从内存里捞出这个类的对象，然后是后面一部分<br><code>--express</code> 就是执行表达式，这里的表达式，<br><code>instances[0].queryName("1")</code> 其中 <code>instances</code> 就是前面从内存中获取的对象数组，因为这些是对象的非静态方法，那就需要从其中取一个来执行我们的方法<br>另外假如我们的场景里需要对比如返回结果做个json序列化<br>我们可以这样</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">vmtool --action getInstances --className com.nicksxs.spbdemo.service.DemoServiceImpl --express '@com.alibaba.fastjson.JSON@toJSONString(instances[0].queryName("1"))'</span><br></pre></td></tr></table></figure>
|
|
<p>这里为什么类开头跟方法开头要用 <code>@</code>, 是因为对于类和静态方法的调用规则是这样，还有如果代码比较多，有可能默认的类加载器中没有加载这个<code>JSON</code>类，那么就需要在参数中加上指定的classloader，<br>可以用sc命令来查找我们的目标类的类加载器，一般来说如果目标类是我们核心业务的，大概率也会有JSON这个类</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sc -d com.nicksxs.spbdemo.service.DemoServiceImpl</span><br></pre></td></tr></table></figure>
|
|
<p>然后在上面命令中加上sc结果中的 classLoaderHash 的值，</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">vmtool --action getInstances -c 18b4aac2 --className com.nicksxs.spbdemo.service.DemoServiceImpl --express '@com.alibaba.fastjson.JSON@toJSONString(instances[0].queryName("1"))'</span><br></pre></td></tr></table></figure>
|
|
<p>这样就能正常执行了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>基于Caddy的快速反向代理</title>
|
|
<url>/2024/10/06/%E5%9F%BA%E4%BA%8ECaddy%E7%9A%84%E5%BF%AB%E9%80%9F%E5%8F%8D%E5%90%91%E4%BB%A3%E7%90%86/</url>
|
|
<content><![CDATA[<p>首先是安装Caddy，</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl</span><br><span class="line">curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg</span><br><span class="line">curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list</span><br><span class="line">sudo apt update</span><br><span class="line">sudo apt install caddy</span><br></pre></td></tr></table></figure>
|
|
<p>安装完成就会自动启动Caddy，<br>这时我们就可以来实践反向代理，并且自带https，</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">caddy reverse-proxy --from :2080 --to :9000</span><br></pre></td></tr></table></figure>
|
|
<p>这样就可以构建一个2080到9000的反向代理<br>如果需要做域名的反向代理<br>可以这样</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">caddy reverse-proxy --from example.com --to :9000</span><br></pre></td></tr></table></figure>
|
|
<p>可以在本地开启一个9000端口的http服务</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">php -S localhost:9000</span><br></pre></td></tr></table></figure>
|
|
<p>然后请求本地</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">curl -v https://localhost</span><br></pre></td></tr></table></figure>
|
|
<p>就能看到响应</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">* Trying 127.0.0.1:443...</span><br><span class="line">* Connected to localhost (127.0.0.1) port 443 (#0)</span><br><span class="line">* ALPN, offering h2</span><br><span class="line">* ALPN, offering http/1.1</span><br><span class="line">* CAfile: /etc/ssl/certs/ca-certificates.crt</span><br><span class="line">* CApath: /etc/ssl/certs</span><br><span class="line">* TLSv1.0 (OUT), TLS header, Certificate Status (22):</span><br><span class="line">* TLSv1.3 (OUT), TLS handshake, Client hello (1):</span><br><span class="line">* TLSv1.2 (IN), TLS header, Certificate Status (22):</span><br><span class="line">* TLSv1.3 (IN), TLS handshake, Server hello (2):</span><br><span class="line">* TLSv1.2 (IN), TLS header, Finished (20):</span><br><span class="line">* TLSv1.2 (IN), TLS header, Supplemental data (23):</span><br><span class="line">* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):</span><br><span class="line">* TLSv1.2 (IN), TLS header, Supplemental data (23):</span><br><span class="line">* TLSv1.3 (IN), TLS handshake, Certificate (11):</span><br><span class="line">* TLSv1.2 (IN), TLS header, Supplemental data (23):</span><br><span class="line">* TLSv1.3 (IN), TLS handshake, CERT verify (15):</span><br><span class="line">* TLSv1.2 (IN), TLS header, Supplemental data (23):</span><br><span class="line">* TLSv1.3 (IN), TLS handshake, Finished (20):</span><br><span class="line">* TLSv1.2 (OUT), TLS header, Finished (20):</span><br><span class="line">* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):</span><br><span class="line">* TLSv1.2 (OUT), TLS header, Supplemental data (23):</span><br><span class="line">* TLSv1.3 (OUT), TLS handshake, Finished (20):</span><br><span class="line">* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256</span><br><span class="line">* ALPN, server accepted to use h2</span><br><span class="line">* Server certificate:</span><br><span class="line">* subject: [NONE]</span><br><span class="line">* start date: Oct 6 12:54:37 2024 GMT</span><br><span class="line">* expire date: Oct 7 00:54:37 2024 GMT</span><br><span class="line">* subjectAltName: host "localhost" matched cert's "localhost"</span><br><span class="line">* issuer: CN=Caddy Local Authority - ECC Intermediate</span><br><span class="line">* SSL certificate verify ok.</span><br><span class="line">* Using HTTP2, server supports multiplexing</span><br><span class="line">* Connection state changed (HTTP/2 confirmed)</span><br><span class="line">* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0</span><br><span class="line">* TLSv1.2 (OUT), TLS header, Supplemental data (23):</span><br><span class="line">* TLSv1.2 (OUT), TLS header, Supplemental data (23):</span><br><span class="line">* TLSv1.2 (OUT), TLS header, Supplemental data (23):</span><br><span class="line">* Using Stream ID: 1 (easy handle 0x55b8f32f5eb0)</span><br><span class="line">* TLSv1.2 (OUT), TLS header, Supplemental data (23):</span><br><span class="line"><span class="meta prompt_">> </span><span class="language-bash">GET / HTTP/2</span></span><br><span class="line"><span class="meta prompt_">> </span><span class="language-bash">Host: localhost</span></span><br><span class="line"><span class="meta prompt_">> </span><span class="language-bash">user-agent: curl/7.81.0</span></span><br><span class="line"><span class="meta prompt_">> </span><span class="language-bash">accept: */*</span></span><br><span class="line"><span class="meta prompt_">></span><span class="language-bash"></span></span><br><span class="line"><span class="language-bash">* TLSv1.2 (IN), TLS header, Supplemental data (23):</span></span><br><span class="line">* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):</span><br><span class="line">* TLSv1.2 (IN), TLS header, Supplemental data (23):</span><br><span class="line">* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!</span><br><span class="line">* TLSv1.2 (OUT), TLS header, Supplemental data (23):</span><br><span class="line">* TLSv1.2 (IN), TLS header, Supplemental data (23):</span><br><span class="line">* TLSv1.2 (IN), TLS header, Supplemental data (23):</span><br><span class="line">< HTTP/2 404</span><br><span class="line">< alt-svc: h3=":443"; ma=2592000</span><br><span class="line">< content-type: text/html; charset=UTF-8</span><br><span class="line">< date: Sun, 06 Oct 2024 13:37:43 GMT</span><br><span class="line">< host: localhost</span><br><span class="line">< server: Caddy</span><br><span class="line">< content-length: 533</span><br><span class="line"><</span><br><span class="line">* TLSv1.2 (IN), TLS header, Supplemental data (23):</span><br><span class="line"><!doctype html><html><head><title>404 Not Found</title><style></span><br><span class="line">body { background-color: #fcfcfc; color: #333333; margin: 0; padding:0; }</span><br><span class="line">h1 { font-size: 1.5em; font-weight: normal; background-color: #9999cc; min-height:2em; line-height:2em; border-bottom: 1px inset black; margin: 0; }</span><br><span class="line">h1, p { padding-left: 10px; }</span><br><span class="line">code.url { background-color: #eeeeee; font-family:monospace; padding:0 2px;}</span><br><span class="line"></style></span><br><span class="line">* Connection #0 to host localhost left intact</span><br><span class="line"></head><body><h1>Not Found</h1><p>The requested resource <code class="url">/</code> was not found on this server.</p></body></html></span><br></pre></td></tr></table></figure>
|
|
<p>然后在php侧服务器就能看到</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">[Sun Oct 6 21:34:29 2024] PHP 8.1.2-1ubuntu2.19 Development Server (http://localhost:9000) started</span><br><span class="line">[Sun Oct 6 21:37:43 2024] 127.0.0.1:38708 Accepted</span><br><span class="line">[Sun Oct 6 21:37:43 2024] 127.0.0.1:38708 [404]: GET / - No such file or directory</span><br><span class="line">[Sun Oct 6 21:37:43 2024] 127.0.0.1:38708 Closing</span><br></pre></td></tr></table></figure>
|
|
<p>还能用配置的形式</p>
|
|
<figure class="highlight json"><table><tr><td class="code"><pre><span class="line"></span><br><span class="line">demo.domain.com <span class="punctuation">{</span></span><br><span class="line"> # 反向代理的地址</span><br><span class="line"> reverse_proxy <span class="number">127.0</span><span class="number">.0</span><span class="number">.1</span><span class="punctuation">:</span>xxxx</span><br><span class="line"><span class="punctuation">}</span></span><br></pre></td></tr></table></figure>
|
|
<p>再运行<code>caddy reload</code>就能启动,还是很方便的</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>小技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>小技巧</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>回归本源理解下mysql的 select for update 锁</title>
|
|
<url>/2025/06/01/%E5%9B%9E%E5%BD%92%E6%9C%AC%E6%BA%90%E7%90%86%E8%A7%A3%E4%B8%8Bmysql%E7%9A%84for-update%E9%94%81/</url>
|
|
<content><![CDATA[<p>很多的业务场景我们会用到锁，包括各种炫酷的分布式锁，但是其实很多情况由于db的可靠性是相对比较高的，所以也可以在适当的情况下使用db来作为锁<br>这里就介绍下比较常用了 select for update 锁</p>
|
|
<h2 id="什么是-SELECT-FOR-UPDATE"><a href="#什么是-SELECT-FOR-UPDATE" class="headerlink" title="什么是 SELECT FOR UPDATE"></a>什么是 SELECT FOR UPDATE</h2><p>SELECT FOR UPDATE 是 MySQL 中一种特殊的查询语句，它在执行查询的同时对选中的行加上排他锁（Exclusive Lock），确保在当前事务结束之前，其他事务无法修改这些数据。这种机制主要用于解决并发环境下的数据一致性问题。</p>
|
|
<h2 id="基本语法"><a href="#基本语法" class="headerlink" title="基本语法"></a>基本语法</h2><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> column1, column2, ... <span class="keyword">FROM</span> table_name <span class="keyword">WHERE</span> <span class="keyword">condition</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<h2 id="工作原理"><a href="#工作原理" class="headerlink" title="工作原理"></a>工作原理</h2><p>当执行 SELECT FOR UPDATE 时，MySQL 会：</p>
|
|
<ol>
|
|
<li><strong>加排他锁</strong>：对查询结果中的每一行数据加上排他锁（X锁）</li>
|
|
<li><strong>阻塞其他事务</strong>：其他事务如果要修改被锁定的行，必须等待当前事务结束</li>
|
|
<li><strong>事务结束释放</strong>：当前事务提交（COMMIT）或回滚（ROLLBACK）时，锁会自动释放</li>
|
|
</ol>
|
|
<h2 id="使用场景"><a href="#使用场景" class="headerlink" title="使用场景"></a>使用场景</h2><h3 id="1-防止超卖问题"><a href="#1-防止超卖问题" class="headerlink" title="1. 防止超卖问题"></a>1. 防止超卖问题</h3><p>电商系统中最常见的应用场景：</p>
|
|
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="comment">-- 开始事务</span></span><br><span class="line"><span class="keyword">START</span> TRANSACTION;</span><br><span class="line"></span><br><span class="line"><span class="comment">-- 查询并锁定商品库存</span></span><br><span class="line"><span class="keyword">SELECT</span> stock <span class="keyword">FROM</span> products <span class="keyword">WHERE</span> id <span class="operator">=</span> <span class="number">1001</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">-- 假设查询结果显示库存为 10</span></span><br><span class="line"><span class="comment">-- 用户购买 3 件商品</span></span><br><span class="line"><span class="keyword">UPDATE</span> products <span class="keyword">SET</span> stock <span class="operator">=</span> stock <span class="operator">-</span> <span class="number">3</span> <span class="keyword">WHERE</span> id <span class="operator">=</span> <span class="number">1001</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">-- 提交事务</span></span><br><span class="line"><span class="keyword">COMMIT</span>;</span><br></pre></td></tr></table></figure>
|
|
<p>这里也可以直接使用start; 开启事务，</p>
|
|
<h3 id="2-生成唯一序列号"><a href="#2-生成唯一序列号" class="headerlink" title="2. 生成唯一序列号"></a>2. 生成唯一序列号</h3><p>确保序列号的唯一性：</p>
|
|
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="keyword">START</span> TRANSACTION;</span><br><span class="line"></span><br><span class="line"><span class="comment">-- 获取当前最大序列号并锁定</span></span><br><span class="line"><span class="keyword">SELECT</span> max_seq <span class="keyword">FROM</span> sequence_table <span class="keyword">WHERE</span> table_name <span class="operator">=</span> <span class="string">'orders'</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">-- 更新序列号</span></span><br><span class="line"><span class="keyword">UPDATE</span> sequence_table <span class="keyword">SET</span> max_seq <span class="operator">=</span> max_seq <span class="operator">+</span> <span class="number">1</span> <span class="keyword">WHERE</span> table_name <span class="operator">=</span> <span class="string">'orders'</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">-- 提交事务</span></span><br><span class="line"><span class="keyword">COMMIT</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<h3 id="3-账户余额操作"><a href="#3-账户余额操作" class="headerlink" title="3. 账户余额操作"></a>3. 账户余额操作</h3><p>银行转账等涉及账户余额的操作：</p>
|
|
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="keyword">START</span> TRANSACTION;</span><br><span class="line"></span><br><span class="line"><span class="comment">-- 锁定转出账户</span></span><br><span class="line"><span class="keyword">SELECT</span> balance <span class="keyword">FROM</span> accounts <span class="keyword">WHERE</span> account_id <span class="operator">=</span> <span class="string">'A001'</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">-- 锁定转入账户</span></span><br><span class="line"><span class="keyword">SELECT</span> balance <span class="keyword">FROM</span> accounts <span class="keyword">WHERE</span> account_id <span class="operator">=</span> <span class="string">'B001'</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">-- 执行转账操作</span></span><br><span class="line"><span class="keyword">UPDATE</span> accounts <span class="keyword">SET</span> balance <span class="operator">=</span> balance <span class="operator">-</span> <span class="number">1000</span> <span class="keyword">WHERE</span> account_id <span class="operator">=</span> <span class="string">'A001'</span>;</span><br><span class="line"><span class="keyword">UPDATE</span> accounts <span class="keyword">SET</span> balance <span class="operator">=</span> balance <span class="operator">+</span> <span class="number">1000</span> <span class="keyword">WHERE</span> account_id <span class="operator">=</span> <span class="string">'B001'</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">COMMIT</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<h2 id="锁的范围和类型"><a href="#锁的范围和类型" class="headerlink" title="锁的范围和类型"></a>锁的范围和类型</h2><h3 id="1-行级锁-vs-表级锁"><a href="#1-行级锁-vs-表级锁" class="headerlink" title="1. 行级锁 vs 表级锁"></a>1. 行级锁 vs 表级锁</h3><ul>
|
|
<li><strong>InnoDB 存储引擎</strong>：使用行级锁，只锁定符合条件的行</li>
|
|
<li><strong>MyISAM 存储引擎</strong>：使用表级锁，锁定整个表</li>
|
|
</ul>
|
|
<h3 id="2-索引对锁范围的影响"><a href="#2-索引对锁范围的影响" class="headerlink" title="2. 索引对锁范围的影响"></a>2. 索引对锁范围的影响</h3><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="comment">-- 情况1：使用主键或唯一索引（精确锁定）</span></span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> users <span class="keyword">WHERE</span> id <span class="operator">=</span> <span class="number">100</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>; <span class="comment">-- 只锁定 id=100 的行</span></span><br><span class="line"></span><br><span class="line"><span class="comment">-- 情况2：使用普通索引（可能锁定多行）</span></span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> users <span class="keyword">WHERE</span> age <span class="operator">=</span> <span class="number">25</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>; <span class="comment">-- 锁定所有 age=25 的行</span></span><br><span class="line"></span><br><span class="line"><span class="comment">-- 情况3：无索引条件（锁定所有行）</span></span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> users <span class="keyword">WHERE</span> name <span class="operator">=</span> <span class="string">'John'</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>; <span class="comment">-- 可能锁定整个表</span></span><br></pre></td></tr></table></figure>
|
|
|
|
<h2 id="死锁问题与预防"><a href="#死锁问题与预防" class="headerlink" title="死锁问题与预防"></a>死锁问题与预防</h2><h3 id="死锁产生的原因"><a href="#死锁产生的原因" class="headerlink" title="死锁产生的原因"></a>死锁产生的原因</h3><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="comment">-- 事务 A</span></span><br><span class="line"><span class="keyword">START</span> TRANSACTION;</span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> table1 <span class="keyword">WHERE</span> id <span class="operator">=</span> <span class="number">1</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>;</span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> table2 <span class="keyword">WHERE</span> id <span class="operator">=</span> <span class="number">2</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>; <span class="comment">-- 等待事务B释放</span></span><br><span class="line"></span><br><span class="line"><span class="comment">-- 事务 B（同时执行）</span></span><br><span class="line"><span class="keyword">START</span> TRANSACTION;</span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> table2 <span class="keyword">WHERE</span> id <span class="operator">=</span> <span class="number">2</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>;</span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> table1 <span class="keyword">WHERE</span> id <span class="operator">=</span> <span class="number">1</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>; <span class="comment">-- 等待事务A释放</span></span><br></pre></td></tr></table></figure>
|
|
<p>这样就会造成死锁等待，注意一点死锁不一定是两个事务相互等待，也可以是循环等待，要达到的是竞态等待</p>
|
|
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="comment">-- 事务 A</span></span><br><span class="line"><span class="keyword">START</span> TRANSACTION;</span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> table1 <span class="keyword">WHERE</span> id <span class="operator">=</span> <span class="number">1</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>;</span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> table2 <span class="keyword">WHERE</span> id <span class="operator">=</span> <span class="number">2</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>; <span class="comment">-- 等待事务B释放</span></span><br><span class="line"></span><br><span class="line"><span class="comment">-- 事务 B（同时执行）</span></span><br><span class="line"><span class="keyword">START</span> TRANSACTION;</span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> table2 <span class="keyword">WHERE</span> id <span class="operator">=</span> <span class="number">2</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>;</span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> table1 <span class="keyword">WHERE</span> id <span class="operator">=</span> <span class="number">3</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>; <span class="comment">-- 等待事务C释放</span></span><br><span class="line"></span><br><span class="line"><span class="comment">-- 事务 C（同时执行）</span></span><br><span class="line"><span class="keyword">START</span> TRANSACTION;</span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> table2 <span class="keyword">WHERE</span> id <span class="operator">=</span> <span class="number">3</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>;</span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> table1 <span class="keyword">WHERE</span> id <span class="operator">=</span> <span class="number">1</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>; <span class="comment">-- 等待事务A释放</span></span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这样也会造成死锁，所以不要把死锁就单纯理解为两个线程互相等待</p>
|
|
<h3 id="预防死锁的策略"><a href="#预防死锁的策略" class="headerlink" title="预防死锁的策略"></a>预防死锁的策略</h3><ol>
|
|
<li><strong>统一加锁顺序</strong>：所有事务按相同顺序获取锁</li>
|
|
<li><strong>减少锁持有时间</strong>：尽快提交或回滚事务</li>
|
|
<li><strong>使用较低的隔离级别</strong>：在业务允许的情况下</li>
|
|
<li><strong>合理设计索引</strong>：避免锁定过多不必要的行</li>
|
|
</ol>
|
|
<h2 id="性能优化建议"><a href="#性能优化建议" class="headerlink" title="性能优化建议"></a>性能优化建议</h2><h3 id="1-合理使用索引"><a href="#1-合理使用索引" class="headerlink" title="1. 合理使用索引"></a>1. 合理使用索引</h3><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="comment">-- 优化前：全表扫描，锁定大量行</span></span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> orders <span class="keyword">WHERE</span> status <span class="operator">=</span> <span class="string">'pending'</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">-- 优化后：为 status 字段添加索引</span></span><br><span class="line"><span class="keyword">ALTER</span> <span class="keyword">TABLE</span> orders <span class="keyword">ADD</span> INDEX idx_status (status);</span><br></pre></td></tr></table></figure>
|
|
|
|
<h3 id="2-缩小锁定范围"><a href="#2-缩小锁定范围" class="headerlink" title="2. 缩小锁定范围"></a>2. 缩小锁定范围</h3><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="comment">-- 避免：锁定所有字段</span></span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> products <span class="keyword">WHERE</span> category <span class="operator">=</span> <span class="string">'electronics'</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">-- 推荐：只选择必要字段</span></span><br><span class="line"><span class="keyword">SELECT</span> id, stock <span class="keyword">FROM</span> products <span class="keyword">WHERE</span> category <span class="operator">=</span> <span class="string">'electronics'</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<h3 id="3-合理的事务边界"><a href="#3-合理的事务边界" class="headerlink" title="3. 合理的事务边界"></a>3. 合理的事务边界</h3><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="comment">-- 不推荐：事务过长</span></span><br><span class="line"><span class="keyword">START</span> TRANSACTION;</span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> table1 <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>;</span><br><span class="line"><span class="comment">-- 大量业务逻辑处理</span></span><br><span class="line"><span class="comment">-- 网络IO操作</span></span><br><span class="line"><span class="keyword">UPDATE</span> table1 <span class="keyword">SET</span> ...;</span><br><span class="line"><span class="keyword">COMMIT</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">-- 推荐：缩短事务时间</span></span><br><span class="line"><span class="comment">-- 先处理业务逻辑</span></span><br><span class="line"><span class="keyword">START</span> TRANSACTION;</span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> table1 <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>;</span><br><span class="line"><span class="keyword">UPDATE</span> table1 <span class="keyword">SET</span> ...;</span><br><span class="line"><span class="keyword">COMMIT</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<h2 id="注意事项和限制"><a href="#注意事项和限制" class="headerlink" title="注意事项和限制"></a>注意事项和限制</h2><h3 id="1-必须在事务中使用"><a href="#1-必须在事务中使用" class="headerlink" title="1. 必须在事务中使用"></a>1. 必须在事务中使用</h3><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="comment">-- 错误：不在事务中，锁立即释放</span></span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> users <span class="keyword">WHERE</span> id <span class="operator">=</span> <span class="number">1</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">-- 正确：在事务中使用</span></span><br><span class="line"><span class="keyword">START</span> TRANSACTION;</span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> users <span class="keyword">WHERE</span> id <span class="operator">=</span> <span class="number">1</span> <span class="keyword">FOR</span> <span class="keyword">UPDATE</span>;</span><br><span class="line"><span class="comment">-- 其他操作</span></span><br><span class="line"><span class="keyword">COMMIT</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<h3 id="2-自动提交模式的影响"><a href="#2-自动提交模式的影响" class="headerlink" title="2. 自动提交模式的影响"></a>2. 自动提交模式的影响</h3><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="comment">-- 检查自动提交状态</span></span><br><span class="line"><span class="keyword">SHOW</span> VARIABLES <span class="keyword">LIKE</span> <span class="string">'autocommit'</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">-- 如果开启了自动提交，需要显式开启事务</span></span><br><span class="line"><span class="keyword">SET</span> autocommit <span class="operator">=</span> <span class="number">0</span>; <span class="comment">-- 关闭自动提交</span></span><br><span class="line"><span class="comment">-- 或者</span></span><br><span class="line"><span class="keyword">START</span> TRANSACTION; <span class="comment">-- 显式开启事务</span></span><br></pre></td></tr></table></figure>
|
|
|
|
<h3 id="3-锁等待超时"><a href="#3-锁等待超时" class="headerlink" title="3. 锁等待超时"></a>3. 锁等待超时</h3><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="comment">-- 设置锁等待超时时间（秒）</span></span><br><span class="line"><span class="keyword">SET</span> innodb_lock_wait_timeout <span class="operator">=</span> <span class="number">60</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<h2 id="替代方案"><a href="#替代方案" class="headerlink" title="替代方案"></a>替代方案</h2><h3 id="1-乐观锁"><a href="#1-乐观锁" class="headerlink" title="1. 乐观锁"></a>1. 乐观锁</h3><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="comment">-- 使用版本号实现乐观锁</span></span><br><span class="line"><span class="keyword">UPDATE</span> products </span><br><span class="line"><span class="keyword">SET</span> stock <span class="operator">=</span> stock <span class="operator">-</span> <span class="number">1</span>, version <span class="operator">=</span> version <span class="operator">+</span> <span class="number">1</span> </span><br><span class="line"><span class="keyword">WHERE</span> id <span class="operator">=</span> <span class="number">1001</span> <span class="keyword">AND</span> version <span class="operator">=</span> <span class="variable">@old_version</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<h3 id="2-原子性操作"><a href="#2-原子性操作" class="headerlink" title="2. 原子性操作"></a>2. 原子性操作</h3><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="comment">-- 直接使用原子性的 UPDATE</span></span><br><span class="line"><span class="keyword">UPDATE</span> products </span><br><span class="line"><span class="keyword">SET</span> stock <span class="operator">=</span> stock <span class="operator">-</span> <span class="number">1</span> </span><br><span class="line"><span class="keyword">WHERE</span> id <span class="operator">=</span> <span class="number">1001</span> <span class="keyword">AND</span> stock <span class="operator">>=</span> <span class="number">1</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<h2 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h2><p>SELECT FOR UPDATE 是 MySQL 中解决并发问题的重要工具，但使用时需要注意：</p>
|
|
<ul>
|
|
<li>合理设计事务边界，避免长时间持有锁</li>
|
|
<li>正确使用索引，减少锁定范围</li>
|
|
<li>统一加锁顺序，预防死锁</li>
|
|
<li>根据业务场景选择合适的并发控制策略</li>
|
|
</ul>
|
|
<p>正确使用 SELECT FOR UPDATE 可以有效保证数据一致性，但也要权衡其对系统性能的影响，在实际应用中需要根据具体业务场景做出合理的选择。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>msyql</category>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>mysql</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>复习下ER图和它的使用方式</title>
|
|
<url>/2025/07/06/%E5%A4%8D%E4%B9%A0%E4%B8%8BER%E5%9B%BE%E5%92%8C%E5%AE%83%E7%9A%84%E4%BD%BF%E7%94%A8%E6%96%B9%E5%BC%8F/</url>
|
|
<content><![CDATA[<p>er图在我们学习数据库的时候是有比较着重的介绍和学习过的，在实际工作中，如果有想对规范的研发流程，那么在技术方案分析设计阶段也需要对系统中涉及到的数据模型用ER图来表示以方便大家理解数据模型的设计和它们之间的关系<br>从维基百科上摘的一个说明和介绍</p>
|
|
<blockquote>
|
|
<p>ER模型，全称为实体联系模型、实体关系模型或实体联系模式图（ERM）（英语：Entity-relationship model）由美籍华人计算机科学家陈品山发明，是概念数据模型的高层描述所使用的数据模型或模式图。<br>常规上的理解ER图分为三个部分，<br>第一个是实体，也就是ER图的E，表示Entity<br>第二个是属性，表示Entity中带有的属性，比如以学生表为例，它有姓名，性别，年龄，年纪等等属性<br>第三个核心的是关系，也就是ER图例的R，表示Relationship，比如学生表跟班级表是怎么样的关系，一般情况下一个学生一定属于一个班级，那么学生和班级的关系就是多对一的关系<br>对于画ER图，比较推荐使用Draw.io来画，开源免费且功能强大<br>它左边菜单栏就自带了ER图的组件<br><img data-src="https://img.nicksxs.me/drawio_menu.png"><br>我们就可以直接拖动组件来画出ER图<br><img data-src="https://img.nicksxs.me/er_demo1.png"><br>这样就能画出简单的ER图了，并且连1对1,1对多这种关系都有现成的连线组件了，真的是治愈强迫症，如果自己画这个N对N的线，老是会觉得画得不一致而感觉画的不好看<br>而且还有一个很强大的功能，比如我们在画ER图的时候已经把表结构设计好了，那么就能直接从表结构生成ER图<br><img data-src="https://img.nicksxs.me/er_from_sql.png"><br>可以在这里导入表结构sql，就能生成对应的实体图了，如果有外键关联还会生成对应的连接，当然手动加一下也很方便了</p>
|
|
</blockquote>
|
|
]]></content>
|
|
<categories>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>技巧</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>基于iflow的qwen模型来体验代码生成</title>
|
|
<url>/2025/11/09/%E5%9F%BA%E4%BA%8Eiflow%E7%9A%84qwen%E6%A8%A1%E5%9E%8B%E6%9D%A5%E4%BD%93%E9%AA%8C%E4%BB%A3%E7%A0%81%E7%94%9F%E6%88%90/</url>
|
|
<content><![CDATA[<p>这里我们用iflow的默认<code>Qwen3-Coder-Plus</code>模型来生成一个todo应用<br>本次的prompt稍微调整了下</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">帮我生成一个todo应用，基于react实现，需要具有非常精美的UI，媲美Instagram那样的网站</span><br></pre></td></tr></table></figure>
|
|
<p>生成的复杂度比之前roocode小很多<br>主要就是在App.tsx里</p>
|
|
<figure class="highlight tsx"><table><tr><td class="code"><pre><span class="line"><span class="keyword">import</span> <span class="title class_">React</span>, { useState, useEffect } <span class="keyword">from</span> <span class="string">'react'</span>;</span><br><span class="line"><span class="keyword">import</span> styled, { createGlobalStyle, keyframes } <span class="keyword">from</span> <span class="string">'styled-components'</span>;</span><br><span class="line"><span class="keyword">import</span> { <span class="title class_">FiCheck</span>, <span class="title class_">FiTrash2</span>, <span class="title class_">FiEdit2</span> } <span class="keyword">from</span> <span class="string">'react-icons/fi'</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 淡入动画</span></span><br><span class="line"><span class="keyword">const</span> fadeIn = keyframes<span class="string">`</span></span><br><span class="line"><span class="string"> from {</span></span><br><span class="line"><span class="string"> opacity: 0;</span></span><br><span class="line"><span class="string"> transform: translateY(20px);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"> to {</span></span><br><span class="line"><span class="string"> opacity: 1;</span></span><br><span class="line"><span class="string"> transform: translateY(0);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 淡入缩放动画</span></span><br><span class="line"><span class="keyword">const</span> fadeInScale = keyframes<span class="string">`</span></span><br><span class="line"><span class="string"> from {</span></span><br><span class="line"><span class="string"> opacity: 0;</span></span><br><span class="line"><span class="string"> transform: scale(0.95);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"> to {</span></span><br><span class="line"><span class="string"> opacity: 1;</span></span><br><span class="line"><span class="string"> transform: scale(1);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 按钮点击动画</span></span><br><span class="line"><span class="keyword">const</span> buttonClick = keyframes<span class="string">`</span></span><br><span class="line"><span class="string"> 0% {</span></span><br><span class="line"><span class="string"> transform: scale(1);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"> 50% {</span></span><br><span class="line"><span class="string"> transform: scale(0.95);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"> 100% {</span></span><br><span class="line"><span class="string"> transform: scale(1);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 按钮悬停脉冲动画</span></span><br><span class="line"><span class="keyword">const</span> pulse = keyframes<span class="string">`</span></span><br><span class="line"><span class="string"> 0% {</span></span><br><span class="line"><span class="string"> box-shadow: 0 0 0 0 rgba(56, 151, 240, 0.4);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"> 70% {</span></span><br><span class="line"><span class="string"> box-shadow: 0 0 0 10px rgba(56, 151, 240, 0);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"> 100% {</span></span><br><span class="line"><span class="string"> box-shadow: 0 0 0 0 rgba(56, 151, 240, 0);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 任务项滑动动画</span></span><br><span class="line"><span class="keyword">const</span> slideIn = keyframes<span class="string">`</span></span><br><span class="line"><span class="string"> from {</span></span><br><span class="line"><span class="string"> opacity: 0;</span></span><br><span class="line"><span class="string"> transform: translateX(30px);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"> to {</span></span><br><span class="line"><span class="string"> opacity: 1;</span></span><br><span class="line"><span class="string"> transform: translateX(0);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 全局样式</span></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">GlobalStyle</span> = createGlobalStyle<span class="string">`</span></span><br><span class="line"><span class="string"> * {</span></span><br><span class="line"><span class="string"> margin: 0;</span></span><br><span class="line"><span class="string"> padding: 0;</span></span><br><span class="line"><span class="string"> box-sizing: border-box;</span></span><br><span class="line"><span class="string"> font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Open Sans', 'Helvetica Neue', sans-serif;</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> body {</span></span><br><span class="line"><span class="string"> background: linear-gradient(135deg, #f5f7fa 0%, #e4edf5 100%);</span></span><br><span class="line"><span class="string"> min-height: 100vh;</span></span><br><span class="line"><span class="string"> padding: 20px;</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 主容器</span></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">AppContainer</span> = styled.<span class="property">div</span><span class="string">`</span></span><br><span class="line"><span class="string"> max-width: 600px;</span></span><br><span class="line"><span class="string"> margin: 0 auto;</span></span><br><span class="line"><span class="string"> background: rgba(255, 255, 255, 0.95);</span></span><br><span class="line"><span class="string"> border-radius: 20px;</span></span><br><span class="line"><span class="string"> box-shadow: 0 10px 30px rgba(0, 0, 0, 0.08);</span></span><br><span class="line"><span class="string"> overflow: hidden;</span></span><br><span class="line"><span class="string"> border: 1px solid rgba(255, 255, 255, 0.2);</span></span><br><span class="line"><span class="string"> backdrop-filter: blur(10px);</span></span><br><span class="line"><span class="string"> animation: <span class="subst">${fadeIn}</span> 0.5s ease-out;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 顶部导航栏</span></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">Header</span> = styled.<span class="property">header</span><span class="string">`</span></span><br><span class="line"><span class="string"> background: linear-gradient(135deg, #6a11cb 0%, #2575fc 100%);</span></span><br><span class="line"><span class="string"> padding: 25px;</span></span><br><span class="line"><span class="string"> text-align: center;</span></span><br><span class="line"><span class="string"> position: relative;</span></span><br><span class="line"><span class="string"> overflow: hidden;</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &::before {</span></span><br><span class="line"><span class="string"> content: '';</span></span><br><span class="line"><span class="string"> position: absolute;</span></span><br><span class="line"><span class="string"> top: -50%;</span></span><br><span class="line"><span class="string"> left: -50%;</span></span><br><span class="line"><span class="string"> width: 200%;</span></span><br><span class="line"><span class="string"> height: 200%;</span></span><br><span class="line"><span class="string"> background: radial-gradient(circle, rgba(255,255,255,0.1) 0%, rgba(255,255,255,0) 70%);</span></span><br><span class="line"><span class="string"> transform: rotate(30deg);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &::after {</span></span><br><span class="line"><span class="string"> content: '';</span></span><br><span class="line"><span class="string"> position: absolute;</span></span><br><span class="line"><span class="string"> bottom: 0;</span></span><br><span class="line"><span class="string"> left: 0;</span></span><br><span class="line"><span class="string"> width: 100%;</span></span><br><span class="line"><span class="string"> height: 2px;</span></span><br><span class="line"><span class="string"> background: linear-gradient(90deg, transparent, rgba(255,255,255,0.3), transparent);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">AppTitle</span> = styled.<span class="property">h1</span><span class="string">`</span></span><br><span class="line"><span class="string"> color: white;</span></span><br><span class="line"><span class="string"> font-size: 2rem;</span></span><br><span class="line"><span class="string"> font-weight: 700;</span></span><br><span class="line"><span class="string"> letter-spacing: 0.5px;</span></span><br><span class="line"><span class="string"> margin: 0;</span></span><br><span class="line"><span class="string"> position: relative;</span></span><br><span class="line"><span class="string"> text-shadow: 0 2px 4px rgba(0, 0, 0, 0.2);</span></span><br><span class="line"><span class="string"> background: linear-gradient(90deg, white, #e0f7fa);</span></span><br><span class="line"><span class="string"> -webkit-background-clip: text;</span></span><br><span class="line"><span class="string"> -webkit-text-fill-color: transparent;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 主内容区域</span></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">MainContent</span> = styled.<span class="property">main</span><span class="string">`</span></span><br><span class="line"><span class="string"> padding: 30px;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 添加任务表单</span></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">AddTodoForm</span> = styled.<span class="property">div</span><span class="string">`</span></span><br><span class="line"><span class="string"> display: flex;</span></span><br><span class="line"><span class="string"> margin-bottom: 25px;</span></span><br><span class="line"><span class="string"> gap: 12px;</span></span><br><span class="line"><span class="string"> animation: <span class="subst">${fadeIn}</span> 0.5s ease-out;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">TodoInput</span> = styled.<span class="property">input</span><span class="string">`</span></span><br><span class="line"><span class="string"> flex: 1;</span></span><br><span class="line"><span class="string"> padding: 16px 20px;</span></span><br><span class="line"><span class="string"> border: 1px solid rgba(134, 142, 150, 0.2);</span></span><br><span class="line"><span class="string"> border-radius: 12px;</span></span><br><span class="line"><span class="string"> font-size: 1rem;</span></span><br><span class="line"><span class="string"> outline: none;</span></span><br><span class="line"><span class="string"> transition: all 0.3s cubic-bezier(0.25, 0.8, 0.25, 1);</span></span><br><span class="line"><span class="string"> background: rgba(255, 255, 255, 0.8);</span></span><br><span class="line"><span class="string"> box-shadow: inset 0 2px 4px rgba(0, 0, 0, 0.05);</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &:focus {</span></span><br><span class="line"><span class="string"> border-color: #3897f0;</span></span><br><span class="line"><span class="string"> background: white;</span></span><br><span class="line"><span class="string"> box-shadow: 0 0 0 3px rgba(56, 151, 240, 0.2), inset 0 2px 4px rgba(0, 0, 0, 0.05);</span></span><br><span class="line"><span class="string"> transform: translateY(-2px);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &::placeholder {</span></span><br><span class="line"><span class="string"> color: #a0aec0;</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">AddButton</span> = styled.<span class="property">button</span><span class="string">`</span></span><br><span class="line"><span class="string"> background: linear-gradient(135deg, #3897f0 0%, #833ab4 100%);</span></span><br><span class="line"><span class="string"> color: white;</span></span><br><span class="line"><span class="string"> border: none;</span></span><br><span class="line"><span class="string"> border-radius: 12px;</span></span><br><span class="line"><span class="string"> padding: 0 24px;</span></span><br><span class="line"><span class="string"> font-size: 1rem;</span></span><br><span class="line"><span class="string"> font-weight: 600;</span></span><br><span class="line"><span class="string"> cursor: pointer;</span></span><br><span class="line"><span class="string"> display: flex;</span></span><br><span class="line"><span class="string"> align-items: center;</span></span><br><span class="line"><span class="string"> justify-content: center;</span></span><br><span class="line"><span class="string"> transition: all 0.3s cubic-bezier(0.25, 0.8, 0.25, 1);</span></span><br><span class="line"><span class="string"> position: relative;</span></span><br><span class="line"><span class="string"> overflow: hidden;</span></span><br><span class="line"><span class="string"> box-shadow: 0 4px 15px rgba(56, 151, 240, 0.3);</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &:hover {</span></span><br><span class="line"><span class="string"> background: linear-gradient(135deg, #2d7bc4 0%, #6a11cb 100%);</span></span><br><span class="line"><span class="string"> transform: translateY(-2px);</span></span><br><span class="line"><span class="string"> box-shadow: 0 6px 20px rgba(56, 151, 240, 0.4);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &:active {</span></span><br><span class="line"><span class="string"> animation: <span class="subst">${buttonClick}</span> 0.2s ease;</span></span><br><span class="line"><span class="string"> transform: translateY(0);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &:focus {</span></span><br><span class="line"><span class="string"> animation: <span class="subst">${pulse}</span> 1.5s infinite;</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &::before {</span></span><br><span class="line"><span class="string"> content: '';</span></span><br><span class="line"><span class="string"> position: absolute;</span></span><br><span class="line"><span class="string"> top: 0;</span></span><br><span class="line"><span class="string"> left: -100%;</span></span><br><span class="line"><span class="string"> width: 100%;</span></span><br><span class="line"><span class="string"> height: 100%;</span></span><br><span class="line"><span class="string"> background: linear-gradient(90deg, transparent, rgba(255,255,255,0.2), transparent);</span></span><br><span class="line"><span class="string"> transition: 0.5s;</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &:hover::before {</span></span><br><span class="line"><span class="string"> left: 100%;</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 任务列表</span></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">TodoList</span> = styled.<span class="property">div</span><span class="string">`</span></span><br><span class="line"><span class="string"> display: flex;</span></span><br><span class="line"><span class="string"> flex-direction: column;</span></span><br><span class="line"><span class="string"> gap: 15px;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 单个任务项</span></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">TodoItem</span> = styled.<span class="property">div</span><{ <span class="attr">completed</span>: <span class="built_in">boolean</span> }><span class="string">`</span></span><br><span class="line"><span class="string"> display: flex;</span></span><br><span class="line"><span class="string"> align-items: center;</span></span><br><span class="line"><span class="string"> padding: 18px;</span></span><br><span class="line"><span class="string"> background: rgba(255, 255, 255, 0.8);</span></span><br><span class="line"><span class="string"> border-radius: 12px;</span></span><br><span class="line"><span class="string"> border: 1px solid rgba(134, 142, 150, 0.15);</span></span><br><span class="line"><span class="string"> transition: all 0.3s cubic-bezier(0.25, 0.8, 0.25, 1);</span></span><br><span class="line"><span class="string"> box-shadow: 0 2px 8px rgba(0, 0, 0, 0.03);</span></span><br><span class="line"><span class="string"> animation: <span class="subst">${slideIn}</span> 0.3s ease-out;</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> <span class="subst">${props => props.completed && <span class="string">`</span></span></span></span><br><span class="line"><span class="string"><span class="subst"><span class="string"> opacity: 0.8;</span></span></span></span><br><span class="line"><span class="string"><span class="subst"><span class="string"> background: rgba(240, 240, 240, 0.6);</span></span></span></span><br><span class="line"><span class="string"><span class="subst"><span class="string"> `</span>}</span></span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &:hover {</span></span><br><span class="line"><span class="string"> border-color: rgba(134, 142, 150, 0.3);</span></span><br><span class="line"><span class="string"> transform: translateY(-2px);</span></span><br><span class="line"><span class="string"> box-shadow: 0 5px 15px rgba(0, 0, 0, 0.08);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">Checkbox</span> = styled.<span class="property">input</span>.<span class="title function_">attrs</span>({ <span class="attr">type</span>: <span class="string">'checkbox'</span> })<span class="string">`</span></span><br><span class="line"><span class="string"> width: 22px;</span></span><br><span class="line"><span class="string"> height: 22px;</span></span><br><span class="line"><span class="string"> cursor: pointer;</span></span><br><span class="line"><span class="string"> accent-color: #4CAF50;</span></span><br><span class="line"><span class="string"> border-radius: 6px;</span></span><br><span class="line"><span class="string"> border: 2px solid #cbd5e0;</span></span><br><span class="line"><span class="string"> transition: all 0.2s ease;</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &:checked {</span></span><br><span class="line"><span class="string"> border-color: #4CAF50;</span></span><br><span class="line"><span class="string"> background: #4CAF50;</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &:hover {</span></span><br><span class="line"><span class="string"> transform: scale(1.1);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">TodoText</span> = styled.<span class="property">span</span><{ <span class="attr">completed</span>: <span class="built_in">boolean</span> }><span class="string">`</span></span><br><span class="line"><span class="string"> flex: 1;</span></span><br><span class="line"><span class="string"> font-size: 1.1rem;</span></span><br><span class="line"><span class="string"> margin: 0 15px;</span></span><br><span class="line"><span class="string"> transition: all 0.3s ease;</span></span><br><span class="line"><span class="string"> color: #2d3748;</span></span><br><span class="line"><span class="string"> <span class="subst">${props => props.completed && <span class="string">`</span></span></span></span><br><span class="line"><span class="string"><span class="subst"><span class="string"> text-decoration: line-through;</span></span></span></span><br><span class="line"><span class="string"><span class="subst"><span class="string"> color: #a0aec0;</span></span></span></span><br><span class="line"><span class="string"><span class="subst"><span class="string"> `</span>}</span></span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">TodoActions</span> = styled.<span class="property">div</span><span class="string">`</span></span><br><span class="line"><span class="string"> display: flex;</span></span><br><span class="line"><span class="string"> gap: 12px;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">ActionButton</span> = styled.<span class="property">button</span><span class="string">`</span></span><br><span class="line"><span class="string"> background: rgba(255, 255, 255, 0.7);</span></span><br><span class="line"><span class="string"> border: 1px solid rgba(134, 142, 150, 0.2);</span></span><br><span class="line"><span class="string"> cursor: pointer;</span></span><br><span class="line"><span class="string"> font-size: 1.2rem;</span></span><br><span class="line"><span class="string"> width: 40px;</span></span><br><span class="line"><span class="string"> height: 40px;</span></span><br><span class="line"><span class="string"> border-radius: 50%;</span></span><br><span class="line"><span class="string"> display: flex;</span></span><br><span class="line"><span class="string"> align-items: center;</span></span><br><span class="line"><span class="string"> justify-content: center;</span></span><br><span class="line"><span class="string"> transition: all 0.2s cubic-bezier(0.25, 0.8, 0.25, 1);</span></span><br><span class="line"><span class="string"> color: #718096;</span></span><br><span class="line"><span class="string"> box-shadow: 0 2px 5px rgba(0, 0, 0, 0.05);</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &:hover {</span></span><br><span class="line"><span class="string"> background: white;</span></span><br><span class="line"><span class="string"> color: #4a5568;</span></span><br><span class="line"><span class="string"> border-color: rgba(134, 142, 150, 0.4);</span></span><br><span class="line"><span class="string"> transform: translateY(-2px) scale(1.1);</span></span><br><span class="line"><span class="string"> box-shadow: 0 4px 10px rgba(0, 0, 0, 0.1);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &:active {</span></span><br><span class="line"><span class="string"> transform: scale(0.95);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">CheckButton</span> = <span class="title function_">styled</span>(<span class="title class_">ActionButton</span>)<span class="string">`</span></span><br><span class="line"><span class="string"> &:hover {</span></span><br><span class="line"><span class="string"> color: #4CAF50;</span></span><br><span class="line"><span class="string"> border-color: rgba(76, 175, 80, 0.4);</span></span><br><span class="line"><span class="string"> background: rgba(76, 175, 80, 0.1);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">EditButton</span> = <span class="title function_">styled</span>(<span class="title class_">ActionButton</span>)<span class="string">`</span></span><br><span class="line"><span class="string"> &:hover {</span></span><br><span class="line"><span class="string"> color: #2b6cb0;</span></span><br><span class="line"><span class="string"> border-color: rgba(43, 108, 176, 0.4);</span></span><br><span class="line"><span class="string"> background: rgba(43, 108, 176, 0.1);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">DeleteButton</span> = <span class="title function_">styled</span>(<span class="title class_">ActionButton</span>)<span class="string">`</span></span><br><span class="line"><span class="string"> &:hover {</span></span><br><span class="line"><span class="string"> color: #e53e3e;</span></span><br><span class="line"><span class="string"> border-color: rgba(229, 62, 62, 0.4);</span></span><br><span class="line"><span class="string"> background: rgba(229, 62, 62, 0.1);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 统计信息</span></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">StatsContainer</span> = styled.<span class="property">div</span><span class="string">`</span></span><br><span class="line"><span class="string"> display: flex;</span></span><br><span class="line"><span class="string"> justify-content: space-between;</span></span><br><span class="line"><span class="string"> margin-top: 25px;</span></span><br><span class="line"><span class="string"> padding: 20px;</span></span><br><span class="line"><span class="string"> background: rgba(255, 255, 255, 0.7);</span></span><br><span class="line"><span class="string"> border-radius: 12px;</span></span><br><span class="line"><span class="string"> font-weight: 500;</span></span><br><span class="line"><span class="string"> color: #718096;</span></span><br><span class="line"><span class="string"> border: 1px solid rgba(134, 142, 150, 0.15);</span></span><br><span class="line"><span class="string"> animation: <span class="subst">${fadeInScale}</span> 0.5s ease-out;</span></span><br><span class="line"><span class="string"> box-shadow: 0 2px 8px rgba(0, 0, 0, 0.03);</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">StatItem</span> = styled.<span class="property">div</span><span class="string">`</span></span><br><span class="line"><span class="string"> text-align: center;</span></span><br><span class="line"><span class="string"> padding: 10px 15px;</span></span><br><span class="line"><span class="string"> border-radius: 8px;</span></span><br><span class="line"><span class="string"> transition: all 0.3s ease;</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> &:hover {</span></span><br><span class="line"><span class="string"> background: rgba(255, 255, 255, 0.5);</span></span><br><span class="line"><span class="string"> transform: translateY(-2px);</span></span><br><span class="line"><span class="string"> }</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">StatNumber</span> = styled.<span class="property">div</span><span class="string">`</span></span><br><span class="line"><span class="string"> font-size: 1.5rem;</span></span><br><span class="line"><span class="string"> font-weight: 700;</span></span><br><span class="line"><span class="string"> color: #2d3748;</span></span><br><span class="line"><span class="string"> background: linear-gradient(135deg, #6a11cb 0%, #2575fc 100%);</span></span><br><span class="line"><span class="string"> -webkit-background-clip: text;</span></span><br><span class="line"><span class="string"> -webkit-text-fill-color: transparent;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">StatLabel</span> = styled.<span class="property">div</span><span class="string">`</span></span><br><span class="line"><span class="string"> font-size: 0.85rem;</span></span><br><span class="line"><span class="string"> color: #a0aec0;</span></span><br><span class="line"><span class="string"> margin-top: 3px;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 空状态</span></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">EmptyState</span> = styled.<span class="property">div</span><span class="string">`</span></span><br><span class="line"><span class="string"> text-align: center;</span></span><br><span class="line"><span class="string"> padding: 60px 20px;</span></span><br><span class="line"><span class="string"> color: #a0aec0;</span></span><br><span class="line"><span class="string"> animation: <span class="subst">${fadeIn}</span> 0.5s ease-out;</span></span><br><span class="line"><span class="string"> background: rgba(255, 255, 255, 0.5);</span></span><br><span class="line"><span class="string"> border-radius: 12px;</span></span><br><span class="line"><span class="string"> margin-top: 10px;</span></span><br><span class="line"><span class="string"> backdrop-filter: blur(5px);</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">EmptyIcon</span> = styled.<span class="property">div</span><span class="string">`</span></span><br><span class="line"><span class="string"> font-size: 3.5rem;</span></span><br><span class="line"><span class="string"> margin-bottom: 20px;</span></span><br><span class="line"><span class="string"> opacity: 0.6;</span></span><br><span class="line"><span class="string"> filter: grayscale(100%);</span></span><br><span class="line"><span class="string"> animation: <span class="subst">${fadeIn}</span> 1s ease-in-out infinite alternate;</span></span><br><span class="line"><span class="string">`</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">interface</span> <span class="title class_">Todo</span> {</span><br><span class="line"> <span class="attr">id</span>: <span class="built_in">string</span>;</span><br><span class="line"> <span class="attr">text</span>: <span class="built_in">string</span>;</span><br><span class="line"> <span class="attr">completed</span>: <span class="built_in">boolean</span>;</span><br><span class="line"> <span class="attr">createdAt</span>: <span class="title class_">Date</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">App</span>: <span class="title class_">React</span>.<span class="property">FC</span> = <span class="function">() =></span> {</span><br><span class="line"> <span class="keyword">const</span> [todos, setTodos] = useState<<span class="title class_">Todo</span>[]>([]);</span><br><span class="line"> <span class="keyword">const</span> [newTodo, setNewTodo] = useState<<span class="built_in">string</span>>(<span class="string">''</span>);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 从localStorage加载数据</span></span><br><span class="line"> <span class="title function_">useEffect</span>(<span class="function">() =></span> {</span><br><span class="line"> <span class="keyword">const</span> savedTodos = <span class="variable language_">localStorage</span>.<span class="title function_">getItem</span>(<span class="string">'todos'</span>);</span><br><span class="line"> <span class="keyword">if</span> (savedTodos) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">const</span> parsedTodos = <span class="title class_">JSON</span>.<span class="title function_">parse</span>(savedTodos);</span><br><span class="line"> <span class="comment">// 将日期字符串转换回Date对象</span></span><br><span class="line"> <span class="keyword">const</span> todosWithDates = parsedTodos.<span class="title function_">map</span>(<span class="function">(<span class="params">todo: <span class="built_in">any</span></span>) =></span> ({</span><br><span class="line"> ...todo,</span><br><span class="line"> <span class="attr">createdAt</span>: <span class="keyword">new</span> <span class="title class_">Date</span>(todo.<span class="property">createdAt</span>)</span><br><span class="line"> }));</span><br><span class="line"> <span class="title function_">setTodos</span>(todosWithDates);</span><br><span class="line"> } <span class="keyword">catch</span> (e) {</span><br><span class="line"> <span class="variable language_">console</span>.<span class="title function_">error</span>(<span class="string">'Error parsing todos from localStorage'</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }, []);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 保存数据到localStorage</span></span><br><span class="line"> <span class="title function_">useEffect</span>(<span class="function">() =></span> {</span><br><span class="line"> <span class="variable language_">localStorage</span>.<span class="title function_">setItem</span>(<span class="string">'todos'</span>, <span class="title class_">JSON</span>.<span class="title function_">stringify</span>(todos));</span><br><span class="line"> }, [todos]);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">const</span> <span class="title function_">addTodo</span> = (<span class="params"></span>) => {</span><br><span class="line"> <span class="keyword">if</span> (newTodo.<span class="title function_">trim</span>() !== <span class="string">''</span>) {</span><br><span class="line"> <span class="keyword">const</span> <span class="attr">newTodoItem</span>: <span class="title class_">Todo</span> = {</span><br><span class="line"> <span class="attr">id</span>: <span class="title class_">Date</span>.<span class="title function_">now</span>().<span class="title function_">toString</span>(),</span><br><span class="line"> <span class="attr">text</span>: newTodo,</span><br><span class="line"> <span class="attr">completed</span>: <span class="literal">false</span>,</span><br><span class="line"> <span class="attr">createdAt</span>: <span class="keyword">new</span> <span class="title class_">Date</span>()</span><br><span class="line"> };</span><br><span class="line"> <span class="title function_">setTodos</span>([newTodoItem, ...todos]); <span class="comment">// 将新任务添加到列表顶部</span></span><br><span class="line"> <span class="title function_">setNewTodo</span>(<span class="string">''</span>);</span><br><span class="line"> }</span><br><span class="line"> };</span><br><span class="line"></span><br><span class="line"> <span class="keyword">const</span> <span class="title function_">toggleTodo</span> = (<span class="params">id: <span class="built_in">string</span></span>) => {</span><br><span class="line"> <span class="title function_">setTodos</span>(todos.<span class="title function_">map</span>(<span class="function"><span class="params">todo</span> =></span></span><br><span class="line"> todo.<span class="property">id</span> === id ? { ...todo, <span class="attr">completed</span>: !todo.<span class="property">completed</span> } : todo</span><br><span class="line"> ));</span><br><span class="line"> };</span><br><span class="line"></span><br><span class="line"> <span class="keyword">const</span> <span class="title function_">deleteTodo</span> = (<span class="params">id: <span class="built_in">string</span></span>) => {</span><br><span class="line"> <span class="title function_">setTodos</span>(todos.<span class="title function_">filter</span>(<span class="function"><span class="params">todo</span> =></span> todo.<span class="property">id</span> !== id));</span><br><span class="line"> };</span><br><span class="line"></span><br><span class="line"> <span class="keyword">const</span> <span class="title function_">editTodo</span> = (<span class="params">id: <span class="built_in">string</span>, newText: <span class="built_in">string</span></span>) => {</span><br><span class="line"> <span class="keyword">if</span> (newText.<span class="title function_">trim</span>() !== <span class="string">''</span>) {</span><br><span class="line"> <span class="title function_">setTodos</span>(todos.<span class="title function_">map</span>(<span class="function"><span class="params">todo</span> =></span></span><br><span class="line"> todo.<span class="property">id</span> === id ? { ...todo, <span class="attr">text</span>: newText } : todo</span><br><span class="line"> ));</span><br><span class="line"> }</span><br><span class="line"> };</span><br><span class="line"></span><br><span class="line"> <span class="keyword">const</span> <span class="title function_">handleKeyPress</span> = (<span class="params">e: React.KeyboardEvent<HTMLInputElement></span>) => {</span><br><span class="line"> <span class="keyword">if</span> (e.<span class="property">key</span> === <span class="string">'Enter'</span>) {</span><br><span class="line"> <span class="title function_">addTodo</span>();</span><br><span class="line"> }</span><br><span class="line"> };</span><br><span class="line"></span><br><span class="line"> <span class="keyword">const</span> completedCount = todos.<span class="title function_">filter</span>(<span class="function"><span class="params">todo</span> =></span> todo.<span class="property">completed</span>).<span class="property">length</span>;</span><br><span class="line"> <span class="keyword">const</span> totalCount = todos.<span class="property">length</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> (</span><br><span class="line"> <span class="language-xml"><span class="tag"><></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">GlobalStyle</span> /></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">AppContainer</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">Header</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">AppTitle</span>></span>Instagram风格Todo<span class="tag"></<span class="name">AppTitle</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">Header</span>></span></span></span><br><span class="line"><span class="language-xml"></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">MainContent</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">AddTodoForm</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">TodoInput</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">type</span>=<span class="string">"text"</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">value</span>=<span class="string">{newTodo}</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">onChange</span>=<span class="string">{(e)</span> =></span> setNewTodo(e.target.value)}</span></span><br><span class="line"><span class="language-xml"> onKeyPress={handleKeyPress}</span></span><br><span class="line"><span class="language-xml"> placeholder="添加新任务..."</span></span><br><span class="line"><span class="language-xml"> /></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">AddButton</span> <span class="attr">onClick</span>=<span class="string">{addTodo}</span>></span></span></span><br><span class="line"><span class="language-xml"> 添加</span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">AddButton</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">AddTodoForm</span>></span></span></span><br><span class="line"><span class="language-xml"></span></span><br><span class="line"><span class="language-xml"> {todos.length > 0 ? (</span></span><br><span class="line"><span class="language-xml"> <span class="tag"><></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">TodoList</span>></span></span></span><br><span class="line"><span class="language-xml"> {todos.map(todo => (</span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">TodoItem</span> <span class="attr">key</span>=<span class="string">{todo.id}</span> <span class="attr">completed</span>=<span class="string">{todo.completed}</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">Checkbox</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">checked</span>=<span class="string">{todo.completed}</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">onChange</span>=<span class="string">{()</span> =></span> toggleTodo(todo.id)}</span></span><br><span class="line"><span class="language-xml"> /></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">TodoText</span> <span class="attr">completed</span>=<span class="string">{todo.completed}</span>></span>{todo.text}<span class="tag"></<span class="name">TodoText</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">TodoActions</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">CheckButton</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">onClick</span>=<span class="string">{()</span> =></span> toggleTodo(todo.id)}</span></span><br><span class="line"><span class="language-xml"> title={todo.completed ? "标记为未完成" : "标记为完成"}</span></span><br><span class="line"><span class="language-xml"> ></span></span><br><span class="line"><span class="language-xml"> {React.createElement(FiCheck as any)}</span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">CheckButton</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">EditButton</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">onClick</span>=<span class="string">{()</span> =></span> {</span></span><br><span class="line"><span class="language-xml"> const newText = prompt('编辑任务:', todo.text);</span></span><br><span class="line"><span class="language-xml"> if (newText !== null) {</span></span><br><span class="line"><span class="language-xml"> editTodo(todo.id, newText);</span></span><br><span class="line"><span class="language-xml"> }</span></span><br><span class="line"><span class="language-xml"> }}</span></span><br><span class="line"><span class="language-xml"> title="编辑"</span></span><br><span class="line"><span class="language-xml"> ></span></span><br><span class="line"><span class="language-xml"> {React.createElement(FiEdit2 as any)}</span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">EditButton</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">DeleteButton</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">onClick</span>=<span class="string">{()</span> =></span> deleteTodo(todo.id)}</span></span><br><span class="line"><span class="language-xml"> title="删除"</span></span><br><span class="line"><span class="language-xml"> ></span></span><br><span class="line"><span class="language-xml"> {React.createElement(FiTrash2 as any)}</span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">DeleteButton</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">TodoActions</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">TodoItem</span>></span></span></span><br><span class="line"><span class="language-xml"> ))}</span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">TodoList</span>></span></span></span><br><span class="line"><span class="language-xml"></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatsContainer</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatItem</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatNumber</span>></span>{totalCount}<span class="tag"></<span class="name">StatNumber</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatLabel</span>></span>总任务<span class="tag"></<span class="name">StatLabel</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">StatItem</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatItem</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatNumber</span>></span>{completedCount}<span class="tag"></<span class="name">StatNumber</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatLabel</span>></span>已完成<span class="tag"></<span class="name">StatLabel</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">StatItem</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatItem</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatNumber</span>></span>{totalCount - completedCount}<span class="tag"></<span class="name">StatNumber</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">StatLabel</span>></span>待完成<span class="tag"></<span class="name">StatLabel</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">StatItem</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">StatsContainer</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></></span></span></span><br><span class="line"><span class="language-xml"> ) : (</span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">EmptyState</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">EmptyIcon</span>></span>📝<span class="tag"></<span class="name">EmptyIcon</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">h2</span>></span>还没有任务<span class="tag"></<span class="name">h2</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">p</span>></span>添加你的第一个任务开始吧！<span class="tag"></<span class="name">p</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">EmptyState</span>></span></span></span><br><span class="line"><span class="language-xml"> )}</span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">MainContent</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">AppContainer</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></></span></span></span><br><span class="line"> );</span><br><span class="line">};</span><br><span class="line"></span><br><span class="line"><span class="keyword">export</span> <span class="keyword">default</span> <span class="title class_">App</span>;</span><br></pre></td></tr></table></figure>
|
|
<p>包括了主体代码和样式，但是颜色样式我还不太满意，希望是之前Instagram那种比较素的颜色，又很精致的那种，但是最近又登录看了下，发现他们家也倒退很多，又通过一些修改，感觉好了一些<br><img data-src="https://img.nicksxs.me/uPic/KjP8ep.png"><br>另外在运行过程中把之前说的问题截到图了，这就是我说的广告，也做的太快了<br><img data-src="https://img.nicksxs.me/uPic/vn6CDl.png"><br>好像是我碰到的第一个带有广告的，不过其他家都是需要为api用量付费的，免费的有点代价也正常<br>另外在调整UI的过程中我们需要让iflow在每一步变更前先把当前代码提交，这样方便我们在多次ui调整中选择最佳的版本</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>学习下用Google的agent开发工具Agent Development Kit</title>
|
|
<url>/2025/10/04/%E5%AD%A6%E4%B9%A0%E4%B8%8B%E7%94%A8Google%E7%9A%84agent%E5%BC%80%E5%8F%91%E5%B7%A5%E5%85%B7Agent-Development-Kit/</url>
|
|
<content><![CDATA[<p>学习下用Google的agent开发工具Agent Development Kit简称 ADK，看着名字就不一般，冲着JDK的地位去的哈哈<br>首先我们安装下pom包</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">dependencies</span>></span></span><br><span class="line"> <span class="comment"><!-- The ADK Core dependency --></span></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>com.google.adk<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>google-adk<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>0.2.0<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="comment"><!-- The ADK Dev Web UI to debug your agent (Optional) --></span></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>com.google.adk<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>google-adk-dev<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>0.2.0<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"><span class="tag"></<span class="name">dependencies</span>></span></span><br><span class="line"></span><br><span class="line"><span class="tag"><<span class="name">build</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">plugins</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.apache.maven.plugins<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>maven-compiler-plugin<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>3.14.0<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">compilerArgs</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">arg</span>></span>-parameters<span class="tag"></<span class="name">arg</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">compilerArgs</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">plugins</span>></span></span><br><span class="line"><span class="tag"></<span class="name">build</span>></span></span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这个版本也说明了它还不是个完全体<br>接下去可以生成一个简单的项目结构</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">project_folder/</span><br><span class="line">├── pom.xml (or build.gradle)</span><br><span class="line">├── src/</span><br><span class="line">├── └── main/</span><br><span class="line">│ └── java/</span><br><span class="line">│ └── agents/</span><br><span class="line">│ └── multitool/</span><br><span class="line">└── test/</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后再multitool中创建个java文件，可以叫 MultiToolAgent.java<br>注意这里需要使用jdk 17及以上版本<br>示例代码如下</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">package</span> agents.multitool;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> com.google.adk.agents.BaseAgent;</span><br><span class="line"><span class="keyword">import</span> com.google.adk.agents.LlmAgent;</span><br><span class="line"><span class="keyword">import</span> com.google.adk.events.Event;</span><br><span class="line"><span class="keyword">import</span> com.google.adk.runner.InMemoryRunner;</span><br><span class="line"><span class="keyword">import</span> com.google.adk.sessions.Session;</span><br><span class="line"><span class="keyword">import</span> com.google.adk.tools.Annotations.Schema;</span><br><span class="line"><span class="keyword">import</span> com.google.adk.tools.FunctionTool;</span><br><span class="line"><span class="keyword">import</span> com.google.genai.types.Content;</span><br><span class="line"><span class="keyword">import</span> com.google.genai.types.Part;</span><br><span class="line"><span class="keyword">import</span> io.reactivex.rxjava3.core.Flowable;</span><br><span class="line"><span class="keyword">import</span> java.nio.charset.StandardCharsets;</span><br><span class="line"><span class="keyword">import</span> java.text.Normalizer;</span><br><span class="line"><span class="keyword">import</span> java.time.ZoneId;</span><br><span class="line"><span class="keyword">import</span> java.time.ZonedDateTime;</span><br><span class="line"><span class="keyword">import</span> java.time.format.DateTimeFormatter;</span><br><span class="line"><span class="keyword">import</span> java.util.Map;</span><br><span class="line"><span class="keyword">import</span> java.util.Scanner;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">MultiToolAgent</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="type">String</span> <span class="variable">USER_ID</span> <span class="operator">=</span> <span class="string">"student"</span>;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="type">String</span> <span class="variable">NAME</span> <span class="operator">=</span> <span class="string">"multi_tool_agent"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// The run your agent with Dev UI, the ROOT_AGENT should be a global public static final variable.</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">BaseAgent</span> <span class="variable">ROOT_AGENT</span> <span class="operator">=</span> initAgent();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> BaseAgent <span class="title function_">initAgent</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> LlmAgent.builder()</span><br><span class="line"> .name(NAME)</span><br><span class="line"> .model(<span class="string">"gemini-2.0-flash"</span>)</span><br><span class="line"> .description(<span class="string">"Agent to answer questions about the time and weather in a city."</span>)</span><br><span class="line"> .instruction(</span><br><span class="line"> <span class="string">"You are a helpful agent who can answer user questions about the time and weather"</span></span><br><span class="line"> + <span class="string">" in a city."</span>)</span><br><span class="line"> .tools(</span><br><span class="line"> FunctionTool.create(MultiToolAgent.class, <span class="string">"getCurrentTime"</span>),</span><br><span class="line"> FunctionTool.create(MultiToolAgent.class, <span class="string">"getWeather"</span>))</span><br><span class="line"> .build();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> Map<String, String> <span class="title function_">getCurrentTime</span><span class="params">(</span></span><br><span class="line"><span class="params"> <span class="meta">@Schema(name = "city",</span></span></span><br><span class="line"><span class="meta"><span class="params"> description = "The name of the city for which to retrieve the current time")</span></span></span><br><span class="line"><span class="params"> String city)</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">normalizedCity</span> <span class="operator">=</span></span><br><span class="line"> Normalizer.normalize(city, Normalizer.Form.NFD)</span><br><span class="line"> .trim()</span><br><span class="line"> .toLowerCase()</span><br><span class="line"> .replaceAll(<span class="string">"(\\p{IsM}+|\\p{IsP}+)"</span>, <span class="string">""</span>)</span><br><span class="line"> .replaceAll(<span class="string">"\\s+"</span>, <span class="string">"_"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> ZoneId.getAvailableZoneIds().stream()</span><br><span class="line"> .filter(zid -> zid.toLowerCase().endsWith(<span class="string">"/"</span> + normalizedCity))</span><br><span class="line"> .findFirst()</span><br><span class="line"> .map(</span><br><span class="line"> zid -></span><br><span class="line"> Map.of(</span><br><span class="line"> <span class="string">"status"</span>,</span><br><span class="line"> <span class="string">"success"</span>,</span><br><span class="line"> <span class="string">"report"</span>,</span><br><span class="line"> <span class="string">"The current time in "</span></span><br><span class="line"> + city</span><br><span class="line"> + <span class="string">" is "</span></span><br><span class="line"> + ZonedDateTime.now(ZoneId.of(zid))</span><br><span class="line"> .format(DateTimeFormatter.ofPattern(<span class="string">"HH:mm"</span>))</span><br><span class="line"> + <span class="string">"."</span>))</span><br><span class="line"> .orElse(</span><br><span class="line"> Map.of(</span><br><span class="line"> <span class="string">"status"</span>,</span><br><span class="line"> <span class="string">"error"</span>,</span><br><span class="line"> <span class="string">"report"</span>,</span><br><span class="line"> <span class="string">"Sorry, I don't have timezone information for "</span> + city + <span class="string">"."</span>));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> Map<String, String> <span class="title function_">getWeather</span><span class="params">(</span></span><br><span class="line"><span class="params"> <span class="meta">@Schema(name = "city",</span></span></span><br><span class="line"><span class="meta"><span class="params"> description = "The name of the city for which to retrieve the weather report")</span></span></span><br><span class="line"><span class="params"> String city)</span> {</span><br><span class="line"> <span class="keyword">if</span> (city.toLowerCase().equals(<span class="string">"new york"</span>)) {</span><br><span class="line"> <span class="keyword">return</span> Map.of(</span><br><span class="line"> <span class="string">"status"</span>,</span><br><span class="line"> <span class="string">"success"</span>,</span><br><span class="line"> <span class="string">"report"</span>,</span><br><span class="line"> <span class="string">"The weather in New York is sunny with a temperature of 25 degrees Celsius (77 degrees"</span></span><br><span class="line"> + <span class="string">" Fahrenheit)."</span>);</span><br><span class="line"></span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> Map.of(</span><br><span class="line"> <span class="string">"status"</span>, <span class="string">"error"</span>, <span class="string">"report"</span>, <span class="string">"Weather information for "</span> + city + <span class="string">" is not available."</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="type">InMemoryRunner</span> <span class="variable">runner</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">InMemoryRunner</span>(ROOT_AGENT);</span><br><span class="line"></span><br><span class="line"> <span class="type">Session</span> <span class="variable">session</span> <span class="operator">=</span></span><br><span class="line"> runner</span><br><span class="line"> .sessionService()</span><br><span class="line"> .createSession(NAME, USER_ID)</span><br><span class="line"> .blockingGet();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> (<span class="type">Scanner</span> <span class="variable">scanner</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Scanner</span>(System.in, StandardCharsets.UTF_8)) {</span><br><span class="line"> <span class="keyword">while</span> (<span class="literal">true</span>) {</span><br><span class="line"> System.out.print(<span class="string">"\nYou > "</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">userInput</span> <span class="operator">=</span> scanner.nextLine();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="string">"quit"</span>.equalsIgnoreCase(userInput)) {</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">Content</span> <span class="variable">userMsg</span> <span class="operator">=</span> Content.fromParts(Part.fromText(userInput));</span><br><span class="line"> Flowable<Event> events = runner.runAsync(USER_ID, session.id(), userMsg);</span><br><span class="line"></span><br><span class="line"> System.out.print(<span class="string">"\nAgent > "</span>);</span><br><span class="line"> events.blockingForEach(event -> System.out.println(event.stringifyContent()));</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这个示例非常简单，就是初始化模型，提供基本的提示词，接下去就是两个简单的工具可供使用，提供城市的时间和纽约的天气<br>要运行起来需要申请api key，然后写到环境变量里</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">export GOOGLE_GENAI_USE_VERTEXAI=FALSE</span><br><span class="line">export GOOGLE_API_KEY=PASTE_YOUR_ACTUAL_API_KEY_HERE</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>在这里替换成自己的api key就行<br>然后运行</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">mvn exec:java \</span><br><span class="line"> -Dexec.mainClass="com.google.adk.web.AdkWebServer" \</span><br><span class="line"> -Dexec.args="--adk.agents.source-dir=src/main/java" \</span><br><span class="line"> -Dexec.classpathScope="compile"</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>就可以看到一个简单的运行界面<br><img data-src="https://img.nicksxs.me/uPic/W6SXj1.png"><br>初使用后发现还是比较容易上手，只是后续更深入的还需要好好研究学习</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>寄生虫观后感</title>
|
|
<url>/2020/03/01/%E5%AF%84%E7%94%9F%E8%99%AB%E8%A7%82%E5%90%8E%E6%84%9F/</url>
|
|
<content><![CDATA[<p>寄生虫这部电影在获得奥斯卡之前就有关注了，豆瓣评分很高，一开始看到这个片名以为是像《铁线虫入侵》那种灾难片，后来看到男主，宋康昊，也是老面孔了，从高中时候在学校操场组织看的《汉江怪物》，有点二的感觉，后来在大学寝室电脑上重新看的时候，室友跟我说是韩国国宝级演员，真人不可貌相，感觉是个呆子的形象。</p>
|
|
<p>但是你说这不是个灾难片，而是个反映社会问题的，就业比较容易往这个方向猜，只是剧情会是怎么样的，一时也没啥头绪，后来不知道哪里看了下一个剧情透露，是一个穷人给富人做家教，然后把自己一家都带进富人家，如果是这样的话可能会把这个怎么带进去作为一个主线，不过事实告诉我，这没那么重要，从第一步朋友的介绍，就显得无比顺利，要去当家教了，作为一个穷成这样的人，瞬间转变成一个衣着得体，言行举止都没让富人家看出破绽的延世大学学生，这真的挺难让人理解，所谓江山易改，本性难移，还有就是这人也正好有那么好能力去辅导，并且诡异的是，多惠也是瞬间就喜欢上了男主，多惠跟将男主介绍给她做家教，也就是多惠原来的家教敏赫，应该也有不少的相处时间，这变了有点大了吧，当然这里也可能因为时长需要，如果说这一点是因为时长，那可能我所有的槽点都是因为这个吧，因为我理解的应该是把家里的人如何一步步地带进富人家，这应该是整个剧情的一个需要更多铺垫去克服这个矛盾点，有时候也想过如果我去当导演，是能拍出个啥，没这个机会，可能有也会是很扯淡的，当然这也不能阻拦我谈谈对这个点的一些看法，毕竟评价一台电冰箱不是说我必须得自己会制冷对吧，这大概是我觉得这个电影的第一个槽点，接下去接二连三的，就是我说的这个最核心的矛盾点，不知道谁说过，这种影视剧应该是源自于生活又高于生活，越是好的作品，越要接近生活，这样子才更能有感同身受。</p>
|
|
<p>接下去的点是金基宇介绍金基婷去给多颂当美术家教，这一步又是我理解的败笔吧，就怎么说呢，没什么铺垫，突然从一个社会底层的穷姑娘，转变成一个气场爆表，把富人家太太唬得一愣一愣的，如果说富太太是比较简单无脑的，那富人自己应该是比较有见识而且是做 IT 的，给自己儿子女儿做家教的，查查底细也很正常吧，但是啥都没有，然后呢，她又开始耍司机的心机了，真的是莫名其妙了，司机真的很惨，窈窕淑女君子好逑，而且这个操作也让我摸不着头脑，这是多腹黑并且有经验才会这么操作，脱内裤真的是让我看得一愣愣的，更看得我一愣一愣的，富人竟然也完全按着这个思路去想了，完全没有别的可能呢，甚至可以去查下行车记录仪或者怎样的，或者有没有毛发体液啥的去检验下，毕竟金基婷也乘坐过这辆车，但是最最让我不懂的还是脱内裤这个操作，究竟是什么样的人才会的呢，值得思考。</p>
|
|
<p>金基泽和忠淑的点也是比较奇怪，首先是金基泽，引起最后那个杀人事件的一个由头，大部分观点都是人为朴社长在之前跟老婆啪啪啪的时候说金基泽的身上有股乘地铁的人的味道，简而言之就是穷人的味道，还有去雯光丈夫身下拿钥匙是对金基泽和雯光丈夫身上的味道的鄙夷，可是这个原因真的站不住脚，即使是同样经济水平，如果身上有比较重的异味，背后讨论下，或者闻到了比较重的味道，有不适的表情和动作很正常吧，像雯光丈夫，在地下室里呆了那么久，身上有异味并且比较重太正常了，就跟在厕所呆久了不会觉得味道大，但是从没味道的地方一进有点味道的厕所就会觉得异样，略尴尬的理由；再说忠淑呢，感觉是太厉害了，能胜任这么一家有钱人的各种挑剔的饮食口味要求的保姆职位，也是让人看懵逼了，看到了不禁想到一个问题，这家人开头是那么地穷，不堪，突然转变成这么地像骗子家族，如果有这么好的骗人能力，应该不会到这种地步吧，如果真的是那么穷，没能力，没志气，又怎么会突然变成这么厉害呢，一家人各司其职，把富人家唬得团团转，而这个前提是，这些人的确能胜任这四个位置，这就是我非常不能理解的点。</p>
|
|
<p>然后说回这个标题，寄生虫，不知道是不是翻译过来不准确，如果真的是叫寄生虫的话，这个寄生虫智商未免也太低了，没有像新冠那样机制，致死率低一点，传染能力强一点，潜伏期也能传染，这个寄生虫第一次受到免疫系统的攻击就自爆了；还有呢，作为一个社会比较低层的打工者，乡下人，对这个审题也是不太审的清，是指这一家人是社会的寄生虫，不思进取，并且死的应该，富人是傻白甜，又有钱又善良，这是给有钱人洗地了还是啥，这个奥斯卡真不知道是怎么得的，总觉得奥斯卡，甚至低一点，豆瓣，得奖的，评分高的都是被一群“精英党”把持的，有黑人主角的，得分高；有同性恋的，得分高；结局惨的，得分高；看不懂的，得分高；就像肖申克的救赎，真不知道是哪里好了，最近看了关于明朝那些事的三杨，杨溥的经历应该比这个厉害吧，可是外国人看不懂，就像外国人不懂中国为什么有反分裂国家法，经历了鸦片战争，八国联军，抗日战争等等，其实跟外国对于黑人的权益的问题，因为有南北战争，所以极度重视这个问题，相应的中国也有自己的历史，请理解。</p>
|
|
<p>简而言之我对寄生虫的评分大概 5~6 分吧。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>影评</category>
|
|
<category>2020</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>影评</tag>
|
|
<tag>寄生虫</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>学习下LlamaIndex-初始篇</title>
|
|
<url>/2024/04/21/%E5%AD%A6%E4%B9%A0%E4%B8%8BLlamaIndex-%E5%88%9D%E5%A7%8B%E7%AF%87/</url>
|
|
<content><![CDATA[<p>LlamaIndex 是目前比较新的大模型RAG框架，RAG是指 检索增强生成 (Retrieval-Augmented Generation, RAG)是指在利用大语言模型回答问题之前，先从外部知识库检索相关信息。RAG 被证明能显著提升答案的准确性，并特别是在知识密集型任务上减少模型的错误输出。通过引用信息来源，用户可以核实答案的准确性，从而增强对模型输出的信任。<br>此外，RAG 有助于快速更新知识并引入特定领域的专业知识。<br>RAG 有效结合了大语言模型的参数化知识和非参数化的外部知识库，成为实施大语言模型的关键方法之一。本文概述了 RAG 在大语言模型时代的发展模式，总结了三种模式：初级 RAG、高级 RAG 和模块化 RAG。<br>首先我们可以了解下LlamaIndex的几大模块，<br>使用大型语言模型(LLMs):无论是OpenAI还是任何托管的LLM,亦或是您自己本地运行的模型,LLM都被用于从索引和存储到查询和解析数据的每一个步骤。LlamaIndex提供了大量可靠经过测试的提示,我们也将向您展示如何自定义您自己的提示。<br>LlamaIndex 主要有这五大块能力</p>
|
|
<ul>
|
|
<li>Data connectors</li>
|
|
<li>Data indexes</li>
|
|
<li>Engines</li>
|
|
<li>Data agents</li>
|
|
<li>Application integrations<br><img data-src="https://img.nicksxs.me/blog/kFxOqW.png"></li>
|
|
</ul>
|
|
<p>其中加载部分可以认为是初始的部分，我们可以从各个数据源去获取信息，比如各种文档，或者confluence，甚至api接口，只需要合适的数据加载器，我们就能将其转化为我们要的数据<br>比如我想要将我博客里的Markdown文件作为知识库加载</p>
|
|
<figure class="highlight python"><table><tr><td class="code"><pre><span class="line">reader = SimpleDirectoryReader(</span><br><span class="line"> input_dir=<span class="string">"./somedir/source/_posts/"</span>,</span><br><span class="line"> required_exts=[<span class="string">".md"</span>],</span><br><span class="line"> recursive=<span class="literal">True</span></span><br><span class="line">)</span><br><span class="line">docs = reader.load_data()</span><br></pre></td></tr></table></figure>
|
|
<p>然后由文档来就构建我们的索引</p>
|
|
<figure class="highlight python"><table><tr><td class="code"><pre><span class="line">index = VectorStoreIndex.from_documents(docs)</span><br></pre></td></tr></table></figure>
|
|
<p>但是这里我们需要提前设置好对应的 embedding 模型，这个很重要，如果有条件推荐使用OpenAI的，如果想用本地的可以考虑使用BAAI智源出品的，只是要注意是否支持多语言，如果要使用本地embedding的话需要安装</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">pip install llama-index-embeddings-huggingface</span><br></pre></td></tr></table></figure>
|
|
<p>这里我选了BAAI也就是智源研究所的bge-m3模型，因为这个是支持多语言的，如果是纯英文的可以用他们 <code>BAAI/bge-small-en-v1.5</code></p>
|
|
<figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">from</span> llama_index.core <span class="keyword">import</span> Settings</span><br><span class="line"><span class="keyword">from</span> llama_index.embeddings.huggingface <span class="keyword">import</span> HuggingFaceEmbedding</span><br><span class="line">Settings.llm = Ollama(model=<span class="string">"gemma:7b"</span>, request_timeout=<span class="number">60.0</span>)</span><br><span class="line">Settings.embed_model = HuggingFaceEmbedding(</span><br><span class="line"> model_name=<span class="string">"BAAI/bge-m3"</span></span><br><span class="line">)</span><br><span class="line">index = VectorStoreIndex.from_documents(docs)</span><br><span class="line">query_engine = index.as_query_engine(llm=gemma_7b)</span><br><span class="line">response = query_engine.query(<span class="string">"数据库主从延迟"</span>)</span><br><span class="line"><span class="built_in">print</span>(response)</span><br></pre></td></tr></table></figure>
|
|
<p>另外LLM 我使用了本地部署的 <code>gemma:7b</code>，因为想尝试下本地的链路，需要注意的是一些文档还是用的原来的service_context，在新版本中这个已经被改掉了，都是通过 Settings 来设置，此外还有诸如 <code>SimpleDirectoryReader</code> 这些包也被挪到了 <code>llama_index.core</code> 中，需要注意下<br>然后我们就能使用这种方式来增强LLM，把我们的本地文档作为知识库来搜索，结合了大模型的能力。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>小工周记一</title>
|
|
<url>/2023/03/05/%E5%B0%8F%E5%B7%A5%E5%91%A8%E8%AE%B0%E4%B8%80/</url>
|
|
<content><![CDATA[<p>开始修老房子又可以更新这个<a href="https://nicksxs.me/tags/%E5%B9%B2%E6%B4%BB/">系列</a>了，比较无聊，就是帮着干点零活的记录，这次过去起的比较早，前几天是在翻新瓦片，到这次周六是收尾了，到了的时候先是继续筛了沙子，上周也筛了，就是只筛了一点点，筛沙子的那个像纱窗一样的还是用一扇中空的中间有一根竖档的门钉上铁丝网做的，就是沙子一直放在外面，原来是有袋子装好的，后来是风吹雨打在上面的都已经破掉，还夹杂了很多树叶什么的，需要过下筛，并且前面都是下雨天，沙子都是湿的，不太像我以前看村里有人造房子筛沙子那样，用铲子铲上去就自己都下去的，湿的就是会在一坨，所以需要铲得比较少，然后撒的比较开，这个需要一点经验，然后如果有人一起的话就可以用扫把按住扫一下，这样就会筛得比较有效，不至于都滑下去，沙子本来大部分是可以筛出来的，还有一点就是这种情况网筛需要放得坡度小一点，不然就更容易直接往下调，袋子没破的就不用过筛了，只是湿掉了的是真的重，筛完了那些破掉了的袋子里的沙子，就没有特别的事情要做了，看到大工在那打墙，有一些敲下来的好的砖头就留着，需要削一下上面的混凝土，据大工说现在砖头要七八毛一块了，后面能够重新利用还是挺值钱的，我跟 LD 就用泥刀和铁锹的在那慢慢削，砖头上的泥灰有的比较牢固有的就像直接是沙子，泥刀刮一下就下来了，有的就结合得比较牢固，不过据说以前的砖头工艺上还比较落后，这个房子差不多是三十年前了的，砖头表面都是有点不平，甚至变形，那时候可能砖头是手工烧制的，现在的砖头比较工艺可好多了，不过可能也贵了很多，后来老丈人也过来了，指导了我们拌泥灰，就是水泥和黄沙混合，以前小时候可喜欢玩这个了，可是就也搞不清楚这个是怎么搅拌的，只看见是水泥跟黄沙围城一圈，中间放水，然后一点点搬进去，首先需要先把干的水泥跟黄沙进行混合，具体的比例是老丈人说的，拌的方式有点像堆沙堆，把水泥黄沙铲起来堆起来，就一直要往这个混合堆的尖尖上堆，这样子它自己滑下来能更好地混合，来回两趟就基本混合均匀了，然后就是跟以前看过的，中间扒拉出一个空间放水，然后慢慢把周围的混合好的泥沙推进去，需要注意不要太着急，旁边的推进去太多太快就会漏水出来，一个是会把旁边的地给弄脏，另一个也铲不回水，然后就是推完所有的都混合水了，就铲起来倒一下，再将铲子翻过来捣几下，后面我们就去吃饭了，去了一家叫金记的，又贵又不太好吃的店，就是离得比较近，六七个人只有六七个菜，吃完要四百多，也是有点离谱了。<br>下午是重头戏，其实本来倒没啥事，就说帮忙搞下靠背（就是踢脚线上面到窗台附近的用木头还有其他材料的装饰性的），都撬撬掉，但是真的是有点离谱了，首先是撬棒真的很重，20 斤的重量（网上查的，没有真的查过），抡起来还要用力地铲进去，因为就是破坏性的要把整个都撬掉，对于我这种又没技巧又没力气的非专业选手，抡起撬棍铲两下手就开始痛了，只是也比较犟，不想刚开始弄就说太重了要休息，后面都完全靠的一点倔强劲撑着，看着里面的工艺感觉也是不容易的，直着横着的木条有好多，竖的一整条，每隔三五十公分，横着的就是三五十公分，每根都要用钉子钉起来，然后外层好像是贴上去，在同一个面的开了头之后就能靠着蛮力往下撬，但是到了转角就又要重新开头，而且最上面一根横条跟紧邻的那一块，大概十几公分，是横着的三条钉在一起，真的是大力都出不了奇迹了，用撬棍的一头用力地敲打都很震手，要从下面往上铲进去撬开一点，然后再从上面往下敲打，这里比较重要的是要小心钉子，我这次运气比较好，踩下去已经扎到了，不过正好在脚趾缝里，没有扎到脚，还是要小心的，做完这个我的手真的是差不多废了，上臂的疼痛已经动一下就受不了了，后面有撬下了最下面当踢脚线的小瓷砖，这个房子估计中间修过一次，两批水泥糊的，新的那批粘的特别牢，敲敲打打了半天才下来一点点，锤子敲上去跟一整块石头一样，震手又没有进展，整个搞完，楼上又在敲墙了，下面的灰尘也是从没见过，我一直在那洒水都完全没有缓解，就上去跟 LD 一起拣砖头，手痛到只能抬两块砖头都会痛了。<br>回到家里开始越来越痛，两个手就完全没法动了，应该也是肌肉拉伤了，我这样是没足够的力气也不会什么技巧，像大工说的，他们也累也难，只是为了赚钱，不过他们有了经验跟技巧，会注意怎么使力不容易受伤，怎么样比较省力，还有一点就是即使这么累，他们一般也下午五点半就下班了，真的很累了，至少还有不少时间可以回家休息，而我们的职业呢，就像 LD 说的回家就像住酒店，就只是来洗澡睡个觉，希望能改善吧</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>小技巧</tag>
|
|
<tag>运动</tag>
|
|
<tag>减肥</tag>
|
|
<tag>跑步</tag>
|
|
<tag>干活</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>学习下n8n的搭建和使用</title>
|
|
<url>/2025/08/03/%E5%AD%A6%E4%B9%A0%E4%B8%8Bn8n%E7%9A%84%E6%90%AD%E5%BB%BA%E5%92%8C%E4%BD%BF%E7%94%A8/</url>
|
|
<content><![CDATA[<p>开始研究学习下现在比较流行的工作流画布，用官方的话就是</p>
|
|
<blockquote>
|
|
<p>Flexible AI workflow automation<br>for technical teams<br>根据中文教程的描述，它也可以认为是个低代码的画布工具，首先安装非常简单，咱们先玩玩本地的<br>前提是装有docker<br>然后就三个命令</p>
|
|
</blockquote>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">docker pull n8nio/n8n</span><br><span class="line">docker volume create n8n_data</span><br><span class="line">docker run -it --rm --name n8n -p 5678:5678 -v n8n_data:/home/node/.n8n docker.n8n.io/n8nio/n8n</span><br></pre></td></tr></table></figure>
|
|
<p>就安装好了，本地端口就是5678<br>在浏览器打开 <code>http://localhost:5678/</code> 就可以看到了，我模仿中文的教程来记录下简单的工作流搭建流程，抓取即刻的帖子<br>第一步我们先建第一个节点，这个类似于是开关，怎么触发这个工作流，<br><img data-src="https://img.nicksxs.me/n8n_first.png"><br>这里也能初步看出n8n的强大，可以通过各种方式来触发，我们就先选一个手动触发<br>然后我们还需要能获取http内容的，<br>我们就添加一个http组件，通过搜索http，可以找到http的请求工具<br><img data-src="https://img.nicksxs.me/n8n_http_request.png"><br>这里我们只需要填入地址<br>比如跟教程中的一样 <code>https://m.okjike.com/users/6d3698d6-0970-49a6-a19d-f8d3cfa33a6f</code>，<br>然后就是要把请求的内容解析出来<br>搜索html，我们就可以找到解析html内容的工具<br><img data-src="https://img.nicksxs.me/n8n_html.png"><br>从网页中找到对应的选择器，记得勾选 return array，否则只会取第一个<br><img data-src="https://img.nicksxs.me/n8n_html_selector.png"><br>执行下当前组件，就能看到内容已经被抓下来了<br>接下去要分割下，选择 <code>Data transformation</code> 的工具中的split out<br>然后就把前面的正文和时间和拖进去<br><img data-src="https://img.nicksxs.me/n8n_split_out.png"><br>最后就是导出文件了<br>搜索 xlsx，然后出现convert to file工具，然后把对应的字段拖进去<br><img data-src="https://img.nicksxs.me/n8n_export_xlsx.png"><br>非常直观和方便的就完成了一个简单的工作流搭建，当然这个还没用上AI工具，大模型和MCP等，但是已经把它的很多特点都介绍了，各种工具真的非常好用和傻瓜化<br>没有复杂难搞的配置和调试，对于一些文字处理表格处理真的很方便<br>后续我们再来看下怎么跟大模型接起来，让它更加的强大。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>学习下mysql新版本支持的row_number方法</title>
|
|
<url>/2025/07/27/%E5%AD%A6%E4%B9%A0%E4%B8%8Bmysql%E6%96%B0%E7%89%88%E6%9C%AC%E6%94%AF%E6%8C%81%E7%9A%84row-number%E6%96%B9%E6%B3%95/</url>
|
|
<content><![CDATA[<p>在实际开发中，我们经常遇到需要在分组内进行排序并获取特定排名记录的需求。比如查找每个班级年龄最大的学生，每个部门薪资最高的员工等。这类问题在MySQL 8.0前后有着截然不同的解决方案。</p>
|
|
<h2 id="传统解决方案（MySQL-8-0之前）"><a href="#传统解决方案（MySQL-8-0之前）" class="headerlink" title="传统解决方案（MySQL 8.0之前）"></a>传统解决方案（MySQL 8.0之前）</h2><h3 id="表结构"><a href="#表结构" class="headerlink" title="表结构"></a>表结构</h3><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> IF <span class="keyword">NOT</span> <span class="keyword">EXISTS</span> `students` (</span><br><span class="line"> `id` <span class="type">bigint</span>(<span class="number">32</span>) <span class="keyword">NOT</span> <span class="keyword">NULL</span> AUTO_INCREMENT COMMENT <span class="string">'主键'</span>,</span><br><span class="line"> `name` <span class="type">varchar</span>(<span class="number">64</span>) <span class="keyword">NOT</span> <span class="keyword">NULL</span> COMMENT <span class="string">'姓名'</span>,</span><br><span class="line"> `age` <span class="type">int</span>(<span class="number">11</span>) <span class="keyword">NOT</span> <span class="keyword">NULL</span> COMMENT <span class="string">'年纪'</span>,</span><br><span class="line"> `class` <span class="type">int</span>(<span class="number">11</span>) <span class="keyword">NOT</span> <span class="keyword">NULL</span> COMMENT <span class="string">'班级'</span>,</span><br><span class="line"> `created_at` datetime <span class="keyword">NOT</span> <span class="keyword">NULL</span> COMMENT <span class="string">'创建时间'</span>,</span><br><span class="line"> `updated_at` <span class="type">timestamp</span> <span class="keyword">NOT</span> <span class="keyword">NULL</span> <span class="keyword">DEFAULT</span> <span class="string">'0000-00-00 00:00:00'</span> <span class="keyword">ON</span> <span class="keyword">UPDATE</span> <span class="built_in">current_timestamp</span>() COMMENT <span class="string">'更新时间'</span>,</span><br><span class="line"> <span class="keyword">PRIMARY</span> KEY (`id`)</span><br><span class="line">) ENGINE<span class="operator">=</span>InnoDB AUTO_INCREMENT<span class="operator">=</span><span class="number">5</span> <span class="keyword">DEFAULT</span> CHARSET<span class="operator">=</span>utf8mb4 COMMENT<span class="operator">=</span><span class="string">'学生表'</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<h3 id="业务场景"><a href="#业务场景" class="headerlink" title="业务场景"></a>业务场景</h3><p>假设我们有如下学生数据，我想找到每个班年纪最大的学生，可以用一些联表或者子查询方法</p>
|
|
<table>
|
|
<thead>
|
|
<tr>
|
|
<th>id</th>
|
|
<th>name</th>
|
|
<th>age</th>
|
|
<th>class</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr>
|
|
<td>1</td>
|
|
<td>学生1</td>
|
|
<td>10</td>
|
|
<td>1</td>
|
|
</tr>
|
|
<tr>
|
|
<td>2</td>
|
|
<td>学生2</td>
|
|
<td>10</td>
|
|
<td>2</td>
|
|
</tr>
|
|
<tr>
|
|
<td>3</td>
|
|
<td>学生3</td>
|
|
<td>11</td>
|
|
<td>1</td>
|
|
</tr>
|
|
<tr>
|
|
<td>4</td>
|
|
<td>学生3</td>
|
|
<td>11</td>
|
|
<td>2</td>
|
|
</tr>
|
|
</tbody></table>
|
|
<h3 id="子查询联表"><a href="#子查询联表" class="headerlink" title="子查询联表"></a>子查询联表</h3><p>在MySQL 8.0之前，我们需要使用相对复杂的子查询配合JOIN来实现：</p>
|
|
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> ta.<span class="operator">*</span> </span><br><span class="line"><span class="keyword">FROM</span> students ta </span><br><span class="line"><span class="keyword">JOIN</span> (</span><br><span class="line"> <span class="keyword">SELECT</span> class, <span class="built_in">MAX</span>(age) <span class="keyword">as</span> max_age </span><br><span class="line"> <span class="keyword">FROM</span> students </span><br><span class="line"> <span class="keyword">GROUP</span> <span class="keyword">BY</span> class</span><br><span class="line">) <span class="keyword">as</span> tb </span><br><span class="line"><span class="keyword">ON</span> ta.class <span class="operator">=</span> tb.class <span class="keyword">AND</span> ta.age <span class="operator">=</span> tb.max_age;</span><br></pre></td></tr></table></figure>
|
|
|
|
<p><strong>分析：</strong></p>
|
|
<ol>
|
|
<li><p><strong>子查询阶段</strong>：<code>SELECT class, MAX(age) as max_age FROM students GROUP BY class</code></p>
|
|
<ul>
|
|
<li>按班级分组，找出每个班级的最大年龄</li>
|
|
<li>结果类似：<code>{class: 1, max_age: 20}, {class: 2, max_age: 21}</code></li>
|
|
</ul>
|
|
</li>
|
|
<li><p><strong>JOIN阶段</strong>：将原表与子查询结果关联</p>
|
|
<ul>
|
|
<li>关联条件：<code>ta.class = tb.class AND ta.age = tb.max_age</code></li>
|
|
<li>最终获得每个班级年龄最大的学生完整信息</li>
|
|
</ul>
|
|
</li>
|
|
</ol>
|
|
<p><strong>局限性：</strong></p>
|
|
<ul>
|
|
<li>查询逻辑相对复杂，可读性不佳</li>
|
|
<li>如果存在同班级同年龄的多个学生，会返回多条记录</li>
|
|
<li>性能上需要进行两次表扫描和一次JOIN操作</li>
|
|
</ul>
|
|
<h2 id="现代解决方案（MySQL-8-0-）"><a href="#现代解决方案（MySQL-8-0-）" class="headerlink" title="现代解决方案（MySQL 8.0+）"></a>现代解决方案（MySQL 8.0+）</h2><h3 id="使用窗口函数优化"><a href="#使用窗口函数优化" class="headerlink" title="使用窗口函数优化"></a>使用窗口函数优化</h3><p>MySQL 8.0引入了窗口函数，让这类问题的解决变得更加优雅：</p>
|
|
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> (</span><br><span class="line"> <span class="keyword">SELECT</span> <span class="operator">*</span>, </span><br><span class="line"> <span class="built_in">ROW_NUMBER</span>() <span class="keyword">OVER</span> (<span class="keyword">PARTITION</span> <span class="keyword">BY</span> class <span class="keyword">ORDER</span> <span class="keyword">BY</span> age <span class="keyword">DESC</span>) <span class="keyword">as</span> row_num</span><br><span class="line"> <span class="keyword">FROM</span> students</span><br><span class="line">) t </span><br><span class="line"><span class="keyword">WHERE</span> row_num <span class="operator">=</span> <span class="number">1</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<p><strong>核心概念解析：</strong></p>
|
|
<ol>
|
|
<li><p><strong>PARTITION BY class</strong>：按班级进行分区</p>
|
|
<ul>
|
|
<li>可以理解为逻辑上的分组，但不会像GROUP BY那样聚合数据</li>
|
|
<li>每个分区内部可以独立进行排序和编号</li>
|
|
</ul>
|
|
</li>
|
|
<li><p><strong>ORDER BY age DESC</strong>：在每个分区内按年龄降序排序</p>
|
|
<ul>
|
|
<li>年龄最大的学生排在第一位</li>
|
|
</ul>
|
|
</li>
|
|
<li><p>**ROW_NUMBER()**：为每个分区内的行分配唯一序号</p>
|
|
<ul>
|
|
<li>从1开始递增，即使有相同值也会分配不同序号</li>
|
|
<li>这就解决了传统方法中重复值的问题</li>
|
|
</ul>
|
|
</li>
|
|
<li><p><strong>外层WHERE row_num = 1</strong>：筛选每个分区的第一条记录</p>
|
|
</li>
|
|
</ol>
|
|
<h3 id="其他窗口函数选择"><a href="#其他窗口函数选择" class="headerlink" title="其他窗口函数选择"></a>其他窗口函数选择</h3><p>除了<code>ROW_NUMBER()</code>，还可以根据业务需求选择其他窗口函数：</p>
|
|
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="comment">-- 使用RANK()：相同值会有相同排名，但会跳跃序号</span></span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> (</span><br><span class="line"> <span class="keyword">SELECT</span> <span class="operator">*</span>, </span><br><span class="line"> <span class="built_in">RANK</span>() <span class="keyword">OVER</span> (<span class="keyword">PARTITION</span> <span class="keyword">BY</span> class <span class="keyword">ORDER</span> <span class="keyword">BY</span> age <span class="keyword">DESC</span>) <span class="keyword">as</span> rank_num</span><br><span class="line"> <span class="keyword">FROM</span> students</span><br><span class="line">) t </span><br><span class="line"><span class="keyword">WHERE</span> rank_num <span class="operator">=</span> <span class="number">1</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">-- 使用DENSE_RANK()：相同值有相同排名，但不跳跃序号</span></span><br><span class="line"><span class="keyword">SELECT</span> <span class="operator">*</span> <span class="keyword">FROM</span> (</span><br><span class="line"> <span class="keyword">SELECT</span> <span class="operator">*</span>, </span><br><span class="line"> <span class="built_in">DENSE_RANK</span>() <span class="keyword">OVER</span> (<span class="keyword">PARTITION</span> <span class="keyword">BY</span> class <span class="keyword">ORDER</span> <span class="keyword">BY</span> age <span class="keyword">DESC</span>) <span class="keyword">as</span> dense_rank_num</span><br><span class="line"> <span class="keyword">FROM</span> students</span><br><span class="line">) t </span><br><span class="line"><span class="keyword">WHERE</span> dense_rank_num <span class="operator">=</span> <span class="number">1</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<h2 id="性能对比与优势"><a href="#性能对比与优势" class="headerlink" title="性能对比与优势"></a>性能对比与优势</h2><h3 id="窗口函数的优势："><a href="#窗口函数的优势：" class="headerlink" title="窗口函数的优势："></a>窗口函数的优势：</h3><ol>
|
|
<li><strong>代码简洁性</strong>：逻辑更直观，维护成本更低</li>
|
|
<li><strong>性能优化</strong>：只需一次表扫描，避免了JOIN操作</li>
|
|
<li><strong>功能丰富</strong>：提供多种排名函数应对不同场景</li>
|
|
<li><strong>处理重复值</strong>：<code>ROW_NUMBER()</code>确保每个分组只返回一条记录</li>
|
|
</ol>
|
|
<h3 id="适用场景扩展："><a href="#适用场景扩展：" class="headerlink" title="适用场景扩展："></a>适用场景扩展：</h3><ul>
|
|
<li>Top-N查询：每个分组的前N条记录</li>
|
|
<li>数据去重：基于特定字段的去重逻辑</li>
|
|
<li>百分位计算：使用<code>PERCENT_RANK()</code>等函数</li>
|
|
<li>移动平均：结合<code>ROWS BETWEEN</code>进行滑动窗口计算</li>
|
|
</ul>
|
|
<p>MySQL 8.0的窗口函数为分组排序查询提供了更现代化的解决方案。相比传统的子查询+JOIN方式，窗口函数不仅在语法上更加简洁直观，在性能上也有显著提升。对于需要在分组内进行复杂数据分析的场景，窗口函数已经成为不可或缺的利器。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>mysql</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>mysql</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>将claude用的更有效一些的小窍门</title>
|
|
<url>/2025/10/12/%E5%B0%86claude%E7%94%A8%E7%9A%84%E6%9B%B4%E6%9C%89%E6%95%88%E4%B8%80%E4%BA%9B%E7%9A%84%E5%B0%8F%E7%AA%8D%E9%97%A8/</url>
|
|
<content><![CDATA[<p>claude的模型现在在写代码这块还是比较权威的，只是也别小瞧了本身它的通用能力<br>我们打开类似于gpt，claude经常是以文本形式的提问，再得到文本形式的回答，但是对于我们程序员来说，还有一些用途其实是可以更有效的<br>比如我们可以让claude帮我们画图<br>像我们的uml图，可以用文本化的plantuml或者mermaid等这些工具来画<br>比如我可以让claude给我画个深度学习路径图</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">graph TD</span><br><span class="line"> Start[开始学习神经网络] --> Phase1[第一阶段: 数学基础]</span><br><span class="line"> </span><br><span class="line"> Phase1 --> Math1[线性代数]</span><br><span class="line"> Phase1 --> Math2[微积分]</span><br><span class="line"> Phase1 --> Math3[概率统计]</span><br><span class="line"> </span><br><span class="line"> Math1 --> Math1a["矩阵运算、特征值<br/>📚 3Blue1Brown线性代数系列<br/>🔗 youtube.com/c/3blue1brown"]</span><br><span class="line"> Math2 --> Math2a["偏导数、梯度、链式法则<br/>📚 MIT 18.01单变量微积分<br/>🔗 ocw.mit.edu"]</span><br><span class="line"> Math3 --> Math3a["期望、方差、贝叶斯定理<br/>📚 Probabilistic ML书籍<br/>🔗 probml.github.io"]</span><br><span class="line"> </span><br><span class="line"> Math1a --> Phase2</span><br><span class="line"> Math2a --> Phase2</span><br><span class="line"> Math3a --> Phase2</span><br><span class="line"> </span><br><span class="line"> Phase2[第二阶段: 编程基础] --> Prog1[Python编程]</span><br><span class="line"> Phase2 --> Prog2[NumPy/Pandas]</span><br><span class="line"> Phase2 --> Prog3[数据可视化]</span><br><span class="line"> </span><br><span class="line"> Prog1 --> Prog1a["基础语法、面向对象<br/>📚 Python Crash Course<br/>🔗 nostarch.com/python-crash-course"]</span><br><span class="line"> Prog2 --> Prog2a["数组操作、数据处理<br/>📚 NumPy官方教程<br/>🔗 numpy.org/doc"]</span><br><span class="line"> Prog3 --> Prog3a["Matplotlib、Seaborn<br/>📚 Python Data Science Handbook<br/>🔗 jakevdp.github.io"]</span><br><span class="line"> </span><br><span class="line"> Prog1a --> Phase3</span><br><span class="line"> Prog2a --> Phase3</span><br><span class="line"> Prog3a --> Phase3</span><br><span class="line"> </span><br><span class="line"> Phase3[第三阶段: 机器学习基础] --> ML1[监督学习]</span><br><span class="line"> Phase3 --> ML2[损失函数]</span><br><span class="line"> Phase3 --> ML3[优化算法]</span><br><span class="line"> </span><br><span class="line"> ML1 --> ML1a["线性回归、逻辑回归<br/>📚 Andrew Ng ML课程<br/>🔗 coursera.org/learn/machine-learning"]</span><br><span class="line"> ML2 --> ML2a["MSE、交叉熵<br/>📚 Deep Learning Book Ch5<br/>🔗 deeplearningbook.org"]</span><br><span class="line"> ML3 --> ML3a["梯度下降、SGD<br/>📚 Sebastian Ruder优化综述<br/>🔗 ruder.io/optimizing-gradient-descent"]</span><br><span class="line"> </span><br><span class="line"> ML1a --> Phase4</span><br><span class="line"> ML2a --> Phase4</span><br><span class="line"> ML3a --> Phase4</span><br><span class="line"> </span><br><span class="line"> Phase4[第四阶段: 神经网络基础] --> NN1[感知机]</span><br><span class="line"> Phase4 --> NN2[前馈神经网络]</span><br><span class="line"> Phase4 --> NN3[反向传播]</span><br><span class="line"> </span><br><span class="line"> NN1 --> NN1a["单层感知机、多层感知机<br/>📚 Neural Networks and Deep Learning<br/>🔗 neuralnetworksanddeeplearning.com"]</span><br><span class="line"> NN2 --> NN2a["全连接层、激活函数<br/>📚 Stanford CS231n Lecture 4<br/>🔗 cs231n.stanford.edu"]</span><br><span class="line"> NN3 --> NN3a["链式法则、梯度计算<br/>📚 Backprop Calculus详解<br/>🔗 colah.github.io"]</span><br><span class="line"> </span><br><span class="line"> NN1a --> Phase5</span><br><span class="line"> NN2a --> Phase5</span><br><span class="line"> NN3a --> Phase5</span><br><span class="line"> </span><br><span class="line"> Phase5[第五阶段: 深度学习框架] --> FW1[PyTorch]</span><br><span class="line"> Phase5 --> FW2[TensorFlow/Keras]</span><br><span class="line"> </span><br><span class="line"> FW1 --> FW1a["Tensor操作、自动微分<br/>📚 PyTorch官方教程<br/>🔗 pytorch.org/tutorials"]</span><br><span class="line"> FW2 --> FW2a["模型构建、训练流程<br/>📚 TensorFlow实战<br/>🔗 tensorflow.org/tutorials"]</span><br><span class="line"> </span><br><span class="line"> FW1a --> Phase6</span><br><span class="line"> FW2a --> Phase6</span><br><span class="line"> </span><br><span class="line"> Phase6[第六阶段: 卷积神经网络CNN] --> CNN1[卷积层原理]</span><br><span class="line"> Phase6 --> CNN2[经典架构]</span><br><span class="line"> Phase6 --> CNN3[应用领域]</span><br><span class="line"> </span><br><span class="line"> CNN1 --> CNN1a["卷积、池化、感受野<br/>📚 CS231n Convolutional Networks<br/>🔗 cs231n.github.io/convolutional-networks"]</span><br><span class="line"> CNN2 --> CNN2a["LeNet、AlexNet、VGG<br/>ResNet、Inception、EfficientNet<br/>📚 论文集合: paperswithcode.com<br/>🔗 paperswithcode.com/methods/category/convolutional-neural-networks"]</span><br><span class="line"> CNN3 --> CNN3a["图像分类、目标检测、分割<br/>📚 Computer Vision: Algorithms and Applications<br/>🔗 szeliski.org/Book"]</span><br><span class="line"> </span><br><span class="line"> CNN1a --> Phase7</span><br><span class="line"> CNN2a --> Phase7</span><br><span class="line"> CNN3a --> Phase7</span><br><span class="line"> </span><br><span class="line"> Phase7[第七阶段: 循环神经网络RNN] --> RNN1[RNN基础]</span><br><span class="line"> Phase7 --> RNN2[LSTM/GRU]</span><br><span class="line"> Phase7 --> RNN3[序列建模]</span><br><span class="line"> </span><br><span class="line"> RNN1 --> RNN1a["循环结构、时间展开<br/>📚 Understanding LSTM Networks<br/>🔗 colah.github.io/posts/2015-08-Understanding-LSTMs"]</span><br><span class="line"> RNN2 --> RNN2a["门控机制、长期依赖<br/>📚 Illustrated Guide to LSTM/GRU<br/>🔗 towardsdatascience.com"]</span><br><span class="line"> RNN3 --> RNN3a["时间序列、语言模型<br/>📚 Sequence Models课程<br/>🔗 coursera.org/learn/nlp-sequence-models"]</span><br><span class="line"> </span><br><span class="line"> RNN1a --> Phase8</span><br><span class="line"> RNN2a --> Phase8</span><br><span class="line"> RNN3a --> Phase8</span><br><span class="line"> </span><br><span class="line"> Phase8[第八阶段: 注意力机制与Transformer] --> ATT1[注意力机制]</span><br><span class="line"> Phase8 --> ATT2[Transformer架构]</span><br><span class="line"> Phase8 --> ATT3[预训练模型]</span><br><span class="line"> </span><br><span class="line"> ATT1 --> ATT1a["Self-Attention、Multi-Head<br/>📚 Attention Is All You Need<br/>🔗 arxiv.org/abs/1706.03762"]</span><br><span class="line"> ATT2 --> ATT2a["Encoder-Decoder、位置编码<br/>📚 The Illustrated Transformer<br/>🔗 jalammar.github.io/illustrated-transformer"]</span><br><span class="line"> ATT3 --> ATT3a["BERT、GPT系列、T5<br/>📚 Hugging Face Course<br/>🔗 huggingface.co/course"]</span><br><span class="line"> </span><br><span class="line"> ATT1a --> Phase9</span><br><span class="line"> ATT2a --> Phase9</span><br><span class="line"> ATT3a --> Phase9</span><br><span class="line"> </span><br><span class="line"> Phase9[第九阶段: 生成模型] --> GEN1[自编码器]</span><br><span class="line"> Phase9 --> GEN2[生成对抗网络GAN]</span><br><span class="line"> Phase9 --> GEN3[扩散模型]</span><br><span class="line"> </span><br><span class="line"> GEN1 --> GEN1a["AE、VAE、特征学习<br/>📚 Tutorial on VAE<br/>🔗 arxiv.org/abs/1606.05908"]</span><br><span class="line"> GEN2 --> GEN2a["判别器、生成器、对抗训练<br/>📚 GAN Lab交互式可视化<br/>🔗 poloclub.github.io/ganlab"]</span><br><span class="line"> GEN3 --> GEN3a["DDPM、Stable Diffusion<br/>📚 Diffusion Models教程<br/>🔗 lilianweng.github.io/posts/2021-07-11-diffusion-models"]</span><br><span class="line"> </span><br><span class="line"> GEN1a --> Phase10</span><br><span class="line"> GEN2a --> Phase10</span><br><span class="line"> GEN3a --> Phase10</span><br><span class="line"> </span><br><span class="line"> Phase10[第十阶段: 高级技术] --> ADV1[正则化技术]</span><br><span class="line"> Phase10 --> ADV2[优化技巧]</span><br><span class="line"> Phase10 --> ADV3[模型压缩]</span><br><span class="line"> </span><br><span class="line"> ADV1 --> ADV1a["Dropout、BatchNorm、数据增强<br/>📚 CS231n训练技巧<br/>🔗 cs231n.github.io/neural-networks-2"]</span><br><span class="line"> ADV2 --> ADV2a["Adam、学习率调度、梯度裁剪<br/>📚 An Overview of Optimization<br/>🔗 arxiv.org/abs/1609.04747"]</span><br><span class="line"> ADV3 --> ADV3a["剪枝、量化、知识蒸馏<br/>📚 Model Compression Survey<br/>🔗 arxiv.org/abs/1710.09282"]</span><br><span class="line"> </span><br><span class="line"> ADV1a --> Phase11</span><br><span class="line"> ADV2a --> Phase11</span><br><span class="line"> ADV3a --> Phase11</span><br><span class="line"> </span><br><span class="line"> Phase11[第十一阶段: 实践项目] --> PROJ1[计算机视觉项目]</span><br><span class="line"> Phase11 --> PROJ2[自然语言处理项目]</span><br><span class="line"> Phase11 --> PROJ3[多模态项目]</span><br><span class="line"> </span><br><span class="line"> PROJ1 --> PROJ1a["Kaggle图像竞赛<br/>物体检测系统<br/>📚 PyImageSearch教程<br/>🔗 pyimagesearch.com"]</span><br><span class="line"> PROJ2 --> PROJ2a["文本分类、情感分析<br/>问答系统、对话机器人<br/>📚 实战项目集合<br/>🔗 github.com/dair-ai/ML-Papers-Explained"]</span><br><span class="line"> PROJ3 --> PROJ3a["图像描述、视觉问答<br/>📚 OpenAI CLIP论文<br/>🔗 arxiv.org/abs/2103.00020"]</span><br><span class="line"> </span><br><span class="line"> PROJ1a --> Phase12</span><br><span class="line"> PROJ2a --> Phase12</span><br><span class="line"> PROJ3a --> Phase12</span><br><span class="line"> </span><br><span class="line"> Phase12[第十二阶段: 前沿研究] --> RES1[大语言模型LLM]</span><br><span class="line"> Phase12 --> RES2[强化学习]</span><br><span class="line"> Phase12 --> RES3[神经架构搜索]</span><br><span class="line"> </span><br><span class="line"> RES1 --> RES1a["GPT-4、Claude、Llama系列<br/>📚 LLM综述论文<br/>🔗 arxiv.org/abs/2303.18223"]</span><br><span class="line"> RES2 --> RES2a["DQN、PPO、AlphaGo<br/>📚 Spinning Up in Deep RL<br/>🔗 spinningup.openai.com"]</span><br><span class="line"> RES3 --> RES3a["AutoML、NAS方法<br/>📚 NAS Survey<br/>🔗 arxiv.org/abs/1808.05377"]</span><br><span class="line"> </span><br><span class="line"> RES1a --> End</span><br><span class="line"> RES2a --> End</span><br><span class="line"> RES3a --> End</span><br><span class="line"> </span><br><span class="line"> End[🎓 持续学习与研究]</span><br><span class="line"> </span><br><span class="line"> style Start fill:#e1f5e1</span><br><span class="line"> style Phase1 fill:#fff4e6</span><br><span class="line"> style Phase2 fill:#fff4e6</span><br><span class="line"> style Phase3 fill:#e3f2fd</span><br><span class="line"> style Phase4 fill:#e3f2fd</span><br><span class="line"> style Phase5 fill:#f3e5f5</span><br><span class="line"> style Phase6 fill:#fce4ec</span><br><span class="line"> style Phase7 fill:#fce4ec</span><br><span class="line"> style Phase8 fill:#e0f2f1</span><br><span class="line"> style Phase9 fill:#fff9c4</span><br><span class="line"> style Phase10 fill:#ffebee</span><br><span class="line"> style Phase11 fill:#e8f5e9</span><br><span class="line"> style Phase12 fill:#e1bee7</span><br><span class="line"> style End fill:#c8e6c9</span><br></pre></td></tr></table></figure>
|
|
<p>生成出来的图效果还是挺不错的<br><img data-src="https://img.nicksxs.me/uPic/Z0yZeI.png"><br>另外比如我们想要生成一个电商系统的下单交易流程</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">sequenceDiagram</span><br><span class="line"> actor User as 👤 用户</span><br><span class="line"> participant Web as 🌐 Web/App前端</span><br><span class="line"> participant Gateway as 🚪 API网关</span><br><span class="line"> participant Auth as 🔐 认证服务</span><br><span class="line"> participant Product as 📦 商品服务</span><br><span class="line"> participant Cart as 🛒 购物车服务</span><br><span class="line"> participant Order as 📋 订单服务</span><br><span class="line"> participant Inventory as 📊 库存服务</span><br><span class="line"> participant Coupon as 🎫 优惠券服务</span><br><span class="line"> participant Payment as 💳 支付服务</span><br><span class="line"> participant PayGateway as 💰 支付网关</span><br><span class="line"> participant MQ as 📨 消息队列</span><br><span class="line"> participant WMS as 🏭 仓储系统</span><br><span class="line"> participant Logistics as 🚚 物流系统</span><br><span class="line"> participant Notify as 📧 通知服务</span><br><span class="line"> participant AfterSale as 🔄 售后服务</span><br><span class="line"> participant Finance as 💵 财务系统</span><br><span class="line"></span><br><span class="line"> Note over User,Finance: 📚 参考架构：微服务电商系统<br/>🔗 github.com/macrozheng/mall</span><br><span class="line"></span><br><span class="line"> %% 登录认证阶段</span><br><span class="line"> rect rgb(230, 245, 255)</span><br><span class="line"> Note over User,Auth: 第一阶段：用户认证</span><br><span class="line"> User->>Web: 1. 打开电商平台</span><br><span class="line"> Web->>Gateway: 2. 请求登录页面</span><br><span class="line"> Gateway->>Auth: 3. 验证会话状态</span><br><span class="line"> </span><br><span class="line"> alt 未登录</span><br><span class="line"> Auth-->>Web: 4. 返回登录页面</span><br><span class="line"> User->>Web: 5. 输入账号密码/手机验证码</span><br><span class="line"> Web->>Auth: 6. 提交登录请求</span><br><span class="line"> Note right of Auth: 🔐 JWT Token生成<br/>📚 RFC 7519标准<br/>🔗 jwt.io</span><br><span class="line"> Auth->>Auth: 7. 验证凭证</span><br><span class="line"> Auth-->>Web: 8. 返回Token</span><br><span class="line"> Web->>Web: 9. 存储Token到Cookie/LocalStorage</span><br><span class="line"> else 已登录</span><br><span class="line"> Auth-->>Web: 4. 验证通过</span><br><span class="line"> end</span><br><span class="line"> end</span><br><span class="line"></span><br><span class="line"> %% 商品浏览阶段</span><br><span class="line"> rect rgb(245, 255, 230)</span><br><span class="line"> Note over User,Product: 第二阶段：商品浏览</span><br><span class="line"> User->>Web: 10. 搜索/浏览商品</span><br><span class="line"> Web->>Gateway: 11. 商品查询请求</span><br><span class="line"> Gateway->>Product: 12. 转发查询</span><br><span class="line"> Note right of Product: 🔍 Elasticsearch全文搜索<br/>📚 搜索引擎实战<br/>🔗 elastic.co/guide</span><br><span class="line"> Product->>Product: 13. 从ES/缓存查询</span><br><span class="line"> Product-->>Web: 14. 返回商品列表</span><br><span class="line"> </span><br><span class="line"> User->>Web: 15. 点击商品详情</span><br><span class="line"> Web->>Gateway: 16. 请求商品详情</span><br><span class="line"> Gateway->>Product: 17. 查询商品信息</span><br><span class="line"> Gateway->>Inventory: 18. 查询库存信息</span><br><span class="line"> Note right of Inventory: 💾 Redis缓存+MySQL<br/>📚 缓存穿透/击穿方案<br/>🔗 redis.io/topics/lru-cache</span><br><span class="line"> </span><br><span class="line"> par 并行查询</span><br><span class="line"> Product-->>Gateway: 19a. 返回商品详情</span><br><span class="line"> and</span><br><span class="line"> Inventory-->>Gateway: 19b. 返回库存数量</span><br><span class="line"> end</span><br><span class="line"> Gateway-->>Web: 20. 聚合返回</span><br><span class="line"> end</span><br><span class="line"></span><br><span class="line"> %% 购物车阶段</span><br><span class="line"> rect rgb(255, 245, 230)</span><br><span class="line"> Note over User,Cart: 第三阶段：加入购物车</span><br><span class="line"> User->>Web: 21. 选择规格并加入购物车</span><br><span class="line"> Web->>Gateway: 22. 加购请求</span><br><span class="line"> Gateway->>Cart: 23. 添加到购物车</span><br><span class="line"> Note right of Cart: 🗄️ Redis存储购物车<br/>📚 分布式Session方案<br/>🔗 spring.io/projects/spring-session</span><br><span class="line"> Cart->>Inventory: 24. 验证库存是否充足</span><br><span class="line"> Inventory-->>Cart: 25. 返回库存状态</span><br><span class="line"> Cart->>Cart: 26. 计算商品小计</span><br><span class="line"> Cart-->>Web: 27. 返回购物车数据</span><br><span class="line"> </span><br><span class="line"> User->>Web: 28. 查看购物车</span><br><span class="line"> Web->>Gateway: 29. 获取购物车列表</span><br><span class="line"> Gateway->>Cart: 30. 查询购物车</span><br><span class="line"> Cart->>Product: 31. 批量查询商品最新价格</span><br><span class="line"> Cart->>Coupon: 32. 查询可用优惠券</span><br><span class="line"> Note right of Coupon: 🎫 优惠券系统设计<br/>📚 促销引擎架构<br/>🔗 tech.meituan.com</span><br><span class="line"> </span><br><span class="line"> par 并行查询</span><br><span class="line"> Product-->>Cart: 33a. 返回价格信息</span><br><span class="line"> and</span><br><span class="line"> Coupon-->>Cart: 33b. 返回可用优惠券</span><br><span class="line"> end</span><br><span class="line"> Cart-->>Web: 34. 返回购物车详情</span><br><span class="line"> end</span><br><span class="line"></span><br><span class="line"> %% 订单创建阶段</span><br><span class="line"> rect rgb(255, 240, 245)</span><br><span class="line"> Note over User,Order: 第四阶段：订单创建</span><br><span class="line"> User->>Web: 35. 点击结算</span><br><span class="line"> Web->>Gateway: 36. 进入结算页</span><br><span class="line"> Gateway->>Order: 37. 创建预订单</span><br><span class="line"> </span><br><span class="line"> User->>Web: 38. 选择收货地址/优惠券</span><br><span class="line"> Web->>Gateway: 39. 提交订单</span><br><span class="line"> Gateway->>Order: 40. 创建订单请求</span><br><span class="line"> </span><br><span class="line"> Note right of Order: 🔢 雪花算法生成订单号<br/>📚 分布式ID生成方案<br/>🔗 github.com/twitter/snowflake</span><br><span class="line"> Order->>Order: 41. 生成订单号</span><br><span class="line"> Order->>Coupon: 42. 锁定优惠券</span><br><span class="line"> Order->>Inventory: 43. 预扣减库存</span><br><span class="line"> Note right of Inventory: ⚠️ 分布式锁防超卖<br/>📚 Redis+Lua脚本<br/>🔗 redisson.org</span><br><span class="line"> </span><br><span class="line"> alt 库存充足</span><br><span class="line"> Inventory-->>Order: 44. 扣减成功</span><br><span class="line"> Coupon-->>Order: 45. 锁定成功</span><br><span class="line"> Order->>Order: 46. 订单状态：待支付</span><br><span class="line"> Order-->>Web: 47. 返回订单信息</span><br><span class="line"> else 库存不足</span><br><span class="line"> Inventory-->>Order: 44. 库存不足</span><br><span class="line"> Order-->>Web: 47. 返回库存不足提示</span><br><span class="line"> end</span><br><span class="line"> end</span><br><span class="line"></span><br><span class="line"> %% 支付阶段</span><br><span class="line"> rect rgb(240, 248, 255)</span><br><span class="line"> Note over User,PayGateway: 第五阶段：支付处理</span><br><span class="line"> User->>Web: 48. 选择支付方式</span><br><span class="line"> Web->>Gateway: 49. 发起支付请求</span><br><span class="line"> Gateway->>Payment: 50. 创建支付单</span><br><span class="line"> </span><br><span class="line"> Note right of Payment: 💳 聚合支付设计<br/>📚 支付系统架构<br/>🔗 github.com/Exrick/xpay</span><br><span class="line"> Payment->>Payment: 51. 生成支付单号</span><br><span class="line"> Payment->>Order: 52. 关联订单</span><br><span class="line"> Payment->>PayGateway: 53. 调用支付渠道</span><br><span class="line"> </span><br><span class="line"> alt 支付宝支付</span><br><span class="line"> Note right of PayGateway: 💰 支付宝SDK<br/>📚 开放平台文档<br/>🔗 opendocs.alipay.com</span><br><span class="line"> PayGateway->>PayGateway: 54. 调用支付宝API</span><br><span class="line"> else 微信支付</span><br><span class="line"> Note right of PayGateway: 💚 微信支付API<br/>📚 支付开发文档<br/>🔗 pay.weixin.qq.com</span><br><span class="line"> PayGateway->>PayGateway: 54. 调用微信支付API</span><br><span class="line"> end</span><br><span class="line"> </span><br><span class="line"> PayGateway-->>User: 55. 返回支付页面/二维码</span><br><span class="line"> User->>User: 56. 完成支付操作</span><br><span class="line"> </span><br><span class="line"> Note over PayGateway,Payment: 🔔 异步回调通知</span><br><span class="line"> PayGateway->>Payment: 57. 支付成功回调</span><br><span class="line"> Note right of Payment: 📚 幂等性设计<br/>🔗 martinfowler.com/articles/patterns-of-distributed-systems</span><br><span class="line"> Payment->>Payment: 58. 验证签名并去重</span><br><span class="line"> Payment->>Order: 59. 更新订单状态为已支付</span><br><span class="line"> Payment->>MQ: 60. 发送支付成功消息</span><br><span class="line"> </span><br><span class="line"> par 异步处理</span><br><span class="line"> MQ->>Notify: 61a. 发送通知</span><br><span class="line"> Note right of Notify: 📧 消息推送<br/>📚 RabbitMQ/Kafka<br/>🔗 rabbitmq.com</span><br><span class="line"> Notify->>User: 短信/邮件/Push通知</span><br><span class="line"> and</span><br><span class="line"> MQ->>Order: 61b. 确认订单</span><br><span class="line"> Order->>Order: 更新订单：待发货</span><br><span class="line"> end</span><br><span class="line"> end</span><br><span class="line"></span><br><span class="line"> %% 仓储物流阶段</span><br><span class="line"> rect rgb(232, 245, 233)</span><br><span class="line"> Note over Order,Logistics: 第六阶段：仓储配送</span><br><span class="line"> Order->>WMS: 62. 推送发货指令</span><br><span class="line"> Note right of WMS: 📦 WMS系统设计<br/>📚 仓储管理实践<br/>🔗 github.com/topics/wms</span><br><span class="line"> </span><br><span class="line"> WMS->>WMS: 63. 订单拆分（多仓）</span><br><span class="line"> WMS->>WMS: 64. 波次拣货</span><br><span class="line"> WMS->>WMS: 65. 打包称重</span><br><span class="line"> WMS->>Logistics: 66. 创建物流订单</span><br><span class="line"> </span><br><span class="line"> Note right of Logistics: 🚚 电子面单生成<br/>📚 菜鸟/快递鸟API<br/>🔗 kdniao.com</span><br><span class="line"> Logistics->>Logistics: 67. 生成运单号</span><br><span class="line"> Logistics->>Logistics: 68. 打印电子面单</span><br><span class="line"> Logistics-->>WMS: 69. 返回物流信息</span><br><span class="line"> </span><br><span class="line"> WMS->>Order: 70. 更新订单为已发货</span><br><span class="line"> Order->>MQ: 71. 发送发货消息</span><br><span class="line"> MQ->>Notify: 72. 通知用户已发货</span><br><span class="line"> Notify->>User: 73. 发送物流信息</span><br><span class="line"> </span><br><span class="line"> loop 物流跟踪</span><br><span class="line"> Logistics->>Logistics: 74. 更新物流轨迹</span><br><span class="line"> Note right of Logistics: 🗺️ 物流轨迹追踪<br/>📚 快递100 API<br/>🔗 kuaidi100.com</span><br><span class="line"> Logistics->>Order: 75. 同步物流状态</span><br><span class="line"> Order->>Notify: 76. 推送物流更新</span><br><span class="line"> Notify->>User: 77. 实时物流通知</span><br><span class="line"> end</span><br><span class="line"> </span><br><span class="line"> Logistics->>Order: 78. 签收成功</span><br><span class="line"> Order->>Order: 79. 更新订单：已签收</span><br><span class="line"> Order->>MQ: 80. 发送签收消息</span><br><span class="line"> end</span><br><span class="line"></span><br><span class="line"> %% 订单完成阶段</span><br><span class="line"> rect rgb(255, 243, 224)</span><br><span class="line"> Note over User,Finance: 第七阶段：订单完成</span><br><span class="line"> MQ->>Order: 81. 触发自动确认收货定时器</span><br><span class="line"> Note right of Order: ⏰ 7天自动确认<br/>📚 XXL-Job定时任务<br/>🔗 xuxueli.com/xxl-job</span><br><span class="line"> </span><br><span class="line"> alt 用户主动确认</span><br><span class="line"> User->>Web: 82a. 确认收货</span><br><span class="line"> Web->>Order: 83a. 确认收货请求</span><br><span class="line"> else 超时自动确认</span><br><span class="line"> Order->>Order: 82b. 7天后自动确认</span><br><span class="line"> end</span><br><span class="line"> </span><br><span class="line"> Order->>Order: 84. 订单状态：已完成</span><br><span class="line"> Order->>Finance: 85. 触发结算</span><br><span class="line"> Note right of Finance: 💰 T+1结算<br/>📚 财务对账系统<br/>🔗 accounting-system-design.com</span><br><span class="line"> </span><br><span class="line"> Finance->>Finance: 86. 计算平台佣金</span><br><span class="line"> Finance->>Finance: 87. 生成结算单</span><br><span class="line"> </span><br><span class="line"> User->>Web: 88. 评价商品</span><br><span class="line"> Web->>Product: 89. 提交评价</span><br><span class="line"> Note right of Product: ⭐ UGC内容审核<br/>📚 评价系统设计<br/>🔗 review-system-architecture.com</span><br><span class="line"> Product->>Product: 90. 审核并发布评价</span><br><span class="line"> end</span><br><span class="line"></span><br><span class="line"> %% 售后阶段</span><br><span class="line"> rect rgb(255, 235, 238)</span><br><span class="line"> Note over User,AfterSale: 第八阶段：售后服务（可选）</span><br><span class="line"> opt 需要售后</span><br><span class="line"> User->>Web: 91. 申请退货/换货</span><br><span class="line"> Web->>AfterSale: 92. 创建售后单</span><br><span class="line"> Note right of AfterSale: 🔄 售后工单系统<br/>📚 7天无理由退货<br/>🔗 消费者权益保护法</span><br><span class="line"> </span><br><span class="line"> AfterSale->>AfterSale: 93. 售后审核</span><br><span class="line"> </span><br><span class="line"> alt 审核通过</span><br><span class="line"> AfterSale-->>User: 94. 审核通过，返回退货地址</span><br><span class="line"> User->>Logistics: 95. 寄回商品</span><br><span class="line"> Logistics->>WMS: 96. 商品入库</span><br><span class="line"> WMS->>AfterSale: 97. 确认收货</span><br><span class="line"> AfterSale->>Inventory: 98. 库存回补</span><br><span class="line"> AfterSale->>Payment: 99. 发起退款</span><br><span class="line"> Note right of Payment: 💵 原路退回<br/>📚 退款流程设计<br/>🔗 refund-process-design.com</span><br><span class="line"> Payment->>PayGateway: 100. 调用退款接口</span><br><span class="line"> PayGateway-->>Payment: 101. 退款成功</span><br><span class="line"> Payment->>MQ: 102. 发送退款消息</span><br><span class="line"> MQ->>Notify: 103. 通知用户</span><br><span class="line"> Notify->>User: 104. 退款到账通知</span><br><span class="line"> else 审核拒绝</span><br><span class="line"> AfterSale-->>User: 94. 拒绝售后申请</span><br><span class="line"> end</span><br><span class="line"> end</span><br><span class="line"> end</span><br><span class="line"></span><br><span class="line"> %% 数据分析阶段</span><br><span class="line"> rect rgb(227, 242, 253)</span><br><span class="line"> Note over Order,Finance: 第九阶段：数据统计分析</span><br><span class="line"> Note over Order,Finance: 📊 实时数据大屏<br/>📚 Flink流式计算<br/>🔗 flink.apache.org</span><br><span class="line"> </span><br><span class="line"> par 数据采集</span><br><span class="line"> Order->>MQ: 订单数据埋点</span><br><span class="line"> and</span><br><span class="line"> Payment->>MQ: 支付数据埋点</span><br><span class="line"> and</span><br><span class="line"> Product->>MQ: 商品数据埋点</span><br><span class="line"> end</span><br><span class="line"> </span><br><span class="line"> MQ->>Finance: 数据聚合分析</span><br><span class="line"> Finance->>Finance: 生成报表<br/>（销售额/转化率/ROI等）</span><br><span class="line"> Note right of Finance: 📈 BI系统<br/>📚 数据仓库建设<br/>🔗 github.com/topics/data-warehouse</span><br><span class="line"> end</span><br><span class="line"></span><br><span class="line"> Note over User,Finance: ✅ 完整交易流程结束</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/uPic/1lMpkq.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>屯菜惊魂记</title>
|
|
<url>/2022/04/24/%E5%B1%AF%E8%8F%9C%E6%83%8A%E9%AD%82%E8%AE%B0/</url>
|
|
<content><![CDATA[<p>因某国际大都市的给力表现，昨儿旁边行政区启动应急响应，同事早上就在群里说要去超市买菜了，到了超市人还特别多，由于来的就是我们经常去的那家超市，一方面为了安全，另一方面是怕已经抢不到了，就去了另一家比较远的超市，开车怕没车位就骑了小电驴，还下着小雨，结果到了超市差不多 12 点多，超市里出来的人都是推着一整车一整车的物资，有些比较像我，整箱的泡面，好几提纸巾，还有各种吃的，都是整箱整箱的，进了超市发现结账包括自助结账的都排很长的队，到了蔬菜货架附近，差点哼起那首歌“空空如也~”，新鲜蔬菜基本已被抢空，只剩下一些卖相不太好的土豆番薯之类的，也算是意料之外情理之中了，本来以为这家超市稍微离封控区远一些会空一点，结果就是所谓的某大都市封控了等物资，杭州市是屯了物资等封控，新鲜蔬菜没了我们也只能买点其他的，神奇的是水果基本都在，可能困难时期水果不算必需品了？还是水果基本人人都已经储备了很多，不太能理解，虽然水果还在，但是称重的地方也还有好多人排队，我们采取了并行策略，LD 在那排队，遥控指挥我去拿其他物资，拿了点碱水面，黑米，那黑米的时候还闹了个乌龙，因为前面就是散装鸡蛋的堆货的地方，结果我们以为是在那后面排队，结果称重那个在那散步了，我们还在那排队，看到后面排队，那几个挑的人也该提醒下吧，几个鸡蛋挑了半天，看看人家大妈，直接拿了四盘，看了下牛奶货架也比较空，不过还有致优跟优倍，不过不算很实惠，本来想买，只是后来赶着去结账，就给忘了，称好了黑米去看了下肉，结果肉也没了，都在买猪蹄，我们也不太爱吃猪蹄，就买了点鸡胸肉，整体看起来我们买的东西真的有点格格不入，不买泡面（因为 LD 不让买了），也不屯啥米和鸡蛋，其实鸡蛋已经买了，米也买了，其他的本身冰箱小也放不下太多东西，我是觉得还可能在屯一点这那的，LD 觉得太多了，基本的米面油有了，其他调味品什么也有了。后面就是排队结账，我去排的时候刚好前面一个小伙子跟大妈在争执，大妈说我们差不多时间来的，你要排前面就前面，小伙子有点不高兴，觉得她就是插队，哈哈，平时一般这种剧情都是发生在我身上的，这会看着前面的吵起来还是很开心的，终于有跟我一样较真的人了，有时候总觉得我是个很纠结，很较真的人，但是我现在慢慢认可了这种较真，如果没有人指出来这种是插队行为，是不对的，就会有越来越多的人觉得是可以随意插队的，正确的事应该要坚持，很多情况大家总是觉得多一事不如少一事，鸡毛蒜皮的没什么好计较的，正是这种想法，那么多人才不管任何规则，反而搞得像遵守规则都是傻 X 似的。回到屯物资，后面结账排到队了也没来得及买原来想买的花生牛奶什么的，毕竟那么多人排着队，回家后因为没有蔬菜，结果就只能吃干菜汤和饭了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>囤物资</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>小技巧之-安卓终端工具Termux访问手机文件</title>
|
|
<url>/2024/09/22/%E5%B0%8F%E6%8A%80%E5%B7%A7%E4%B9%8B-%E5%AE%89%E5%8D%93%E7%BB%88%E7%AB%AF%E5%B7%A5%E5%85%B7Termux%E8%AE%BF%E9%97%AE%E6%89%8B%E6%9C%BA%E6%96%87%E4%BB%B6/</url>
|
|
<content><![CDATA[<p>Termux是安卓下一个终端工具，一开始以为就是一些极客的高端玩具，在安卓下编程用，实际的实用性不太强，直到之前稍微研究了下，还真的是个神器，这个神器的原因在于三方面<br>第一点，它其实不只是个终端工具，而是个类似于iterm2 + brew的组合拳神器<br>自带了pkg包管理工具<br><img data-src="https://img.nicksxs.me/blog/%E5%BE%AE%E4%BF%A1%E5%9B%BE%E7%89%87_20240922143821.jpg"><br>还可以通过<code>termux-change-repo</code>来更改包的安装源<br>有了这个工具，我们就可以在手机里运行一些简单的程序，比如跑个php脚本，因为可以安装php<br><img data-src="https://img.nicksxs.me/blog/%E5%BE%AE%E4%BF%A1%E5%9B%BE%E7%89%87_20240922144834.jpg"><br>甚至可以安装nginx，在手机里跑个web应用，可以通过安装git来拉取我们的代码，实现手机开发可能比较难，但是在手机上运行代码还是可以的<br>第二点是这个工具可以开启安卓文件的访问，默认它是限定在应用目录下的，我们可以通过以下命令来请求访问安卓文件目录权限 </p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">termux-setup-storage</span><br></pre></td></tr></table></figure>
|
|
<p>然后我们去到 <code>/storage/emulated/0</code> 目录下，就发现安卓的文件就在这个目录下，举个例子这样我们就可以用脚本来处理我的照片文件<br>第三点是其实目前的手机处理器性能已经到了一个非常不错的阶段，比如我看了下我的渣渣安卓机是高通骁龙870处理器，它是个21年出的手机处理器，性能大致接近于桌面端i5 4590<br>870的geekbench跑分，单核1141，多核3317<br>i5 4590的geekbench跑分，单核1142，3165<br>单核非常接近，多核还稍微领先，并且最近也看到有用最新的高通骁龙8gen3来跑端侧大模型的，其实说明了目前手机的性能真的达到了很高的水平，提供了更多的可玩性，虽然从另一个角度来讲，870在运行现在最新版本的手机应用时会发烫变卡，那是因为手机应用往往要做非常多的事情，后台保活，前台的各种图形渲染，包括微信，支付宝都有了内置的小程序，那么都需要一套资源隔离机制来实现运行态的维护，并且各种内容也搞得花里胡哨都需要资源，我们只是运行简单的脚本，手机的可用性不比很多云服务器差，一般云服务器我们个人用户都买的是比较低配的，最低1核1g的配置，远不如现在的手机强大，可玩性是真的不错的。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>小技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>小技巧</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>小技巧-用iptables统计网速占用</title>
|
|
<url>/2024/09/28/%E5%B0%8F%E6%8A%80%E5%B7%A7-%E7%94%A8iptables%E7%BB%9F%E8%AE%A1%E7%BD%91%E9%80%9F%E5%8D%A0%E7%94%A8/</url>
|
|
<content><![CDATA[<p>目前我在家里用的路由器是个装了ImmortalWrt的NX30 Pro路由器，由于内存只有256兆，默认没带网速统计和限制网速的插件，对于网络速度限制就有点困难，刚好这次网上找到了一个可以用iptables统计网速的脚本，简单记录下</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_"></span></span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash">!/bin/sh</span></span><br><span class="line">echo "Collecting data..."</span><br><span class="line">echo ""</span><br><span class="line">cat /proc/net/arp | grep : | grep ^192 | grep -v 00:00:00:00:00:00| awk '{print $1}'> mac-arp</span><br><span class="line">iptables -N UPLOAD</span><br><span class="line">iptables -N DOWNLOAD</span><br><span class="line">while read line;do iptables -I FORWARD 1 -s $line -j UPLOAD;done < mac-arp</span><br><span class="line">while read line;do iptables -I FORWARD 1 -d $line -j DOWNLOAD;done < mac-arp</span><br><span class="line">sleep 1</span><br><span class="line">echo "Download speed:"</span><br><span class="line">echo ""</span><br><span class="line">iptables -nvx -L FORWARD | grep DOWNLOAD | awk '{print $2/1024/1" KB/s ",$1/10" packets/s", $9}' | sort -n -r</span><br><span class="line">echo ""</span><br><span class="line">echo "Upload speed:"</span><br><span class="line">echo ""</span><br><span class="line">iptables -nvx -L FORWARD | grep UPLOAD | awk '{print $2/1024/1" KB/s ",$1/10" packets/s", $8}' | sort -n -r</span><br><span class="line">while read line;do iptables -D FORWARD -s $line -j UPLOAD;done < mac-arp</span><br><span class="line">while read line;do iptables -D FORWARD -d $line -j DOWNLOAD;done < mac-arp</span><br><span class="line">iptables -X UPLOAD</span><br><span class="line">iptables -X DOWNLOAD</span><br></pre></td></tr></table></figure>
|
|
<p>首先是通过arp记录内网ip地址，然后添加<code>UPLOAD</code>和<code>DOWNLOAD</code>链，<br>然后为mac-arp文件中每个ip地址添加<code>FORWARD</code>规则，将流量导到<code>UPLOAD</code>和<code>DOWNLOAD</code>链<br>等待1秒，统计下载和上传速度，计算KB/s,packets/s，再清理规则<br>然后对于上传占用高的，我们可以用</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">iptables -t mangle -I FORWARD 1 -s 192.168.x.x -m limit --limit 800/s --limit-burst 1000 -j ACCEPT</span><br><span class="line">iptables -t mangle -I FORWARD 2 -s 192.168.x.x -j DROP</span><br></pre></td></tr></table></figure>
|
|
<p>限制通过包的数量为800，不过这个不绝对精确</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>小技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>小技巧</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>尝试下图片向量化</title>
|
|
<url>/2024/07/28/%E5%B0%9D%E8%AF%95%E4%B8%8B%E5%9B%BE%E7%89%87%E5%90%91%E9%87%8F%E5%8C%96/</url>
|
|
<content><![CDATA[<p>之前觉得谷歌的以图搜图很厉害，现在似乎这个路径还毕竟清晰了，首先要有图片库，把它们向量化以后存储起来，然后对于目标图片也做向量化，再做检索<br>那么我们先来做重要的这一步，图片的向量化，因为向量化以后就跟图片没关系了，直接用前面讲到的向量的近似搜索就可以做到以图搜图了<br>这边我们用到了<a href="https://towhee.io/">towhee</a>工具，towhee是个机器学习的pipeline工具，可以做数据源（文件，图片，媒体，文本）–> 模型 –> 向量。<br>首先我们安装towhee</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">pip install towhee towhee.models</span><br></pre></td></tr></table></figure>
|
|
<p>这里有第一个坑，因为有torch依赖，但是目前torch支持的python版本是最高3.9，再往上可能就有问题了<br>所以我们要先创建一个3.9的环境</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">conda create -n py9 python=3.9</span><br></pre></td></tr></table></figure>
|
|
<p>然后我们会遇到第二个坑<br>就是milvus的客户端</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">pip install pymilvus==2.3.0</span><br></pre></td></tr></table></figure>
|
|
<p>再早的客户端会有依赖不支持，更新的客户端会连接不上<br>然后再在milvus创建一个Collection<br><img data-src="https://img.nicksxs.me/blog/PpwX4t.png"><br>然后这个是不对的，因为我使用attu界面创建的，默认第二个字段一定得是向量字段，所以又用了上次的Java代码来创建</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">CreateCollectionReq.<span class="type">CollectionSchema</span> <span class="variable">collectionSchema</span> <span class="operator">=</span> clientV2.createSchema();</span><br><span class="line"><span class="comment">// add two fileds, id and vector</span></span><br><span class="line"> <span class="type">Integer</span> <span class="variable">dim</span> <span class="operator">=</span> <span class="number">2048</span>;</span><br><span class="line"> collectionSchema.addField(AddFieldReq.builder().fieldName(<span class="string">"url"</span>).dataType(DataType.VarChar).build());</span><br><span class="line"> collectionSchema.addField(AddFieldReq.builder().fieldName(<span class="string">"embedding"</span>).dataType(DataType.FloatVector).dimension(dim).build());</span><br><span class="line"> collectionSchema.addField(AddFieldReq.builder().fieldName(<span class="string">"id"</span>).dataType(DataType.Int64).isPrimaryKey(Boolean.TRUE).autoID(Boolean.TRUE).description(<span class="string">"id"</span>).build());</span><br><span class="line"></span><br><span class="line"> <span class="type">CreateCollectionReq</span> <span class="variable">req</span> <span class="operator">=</span> CreateCollectionReq</span><br><span class="line"> .builder()</span><br><span class="line"> .collectionSchema(collectionSchema)</span><br><span class="line"> .collectionName(<span class="string">"text_image_search"</span>)</span><br><span class="line"> .dimension(dim).build();</span><br><span class="line"> clientV2.createCollection(req);</span><br></pre></td></tr></table></figure>
|
|
<p>并且注意字段顺序不能错，这个顺序在towhee的demo中也没地方直接修改<br>看一下towhee的demo<br>地址在这<br><a href="https://towhee.io/tasks/detail/pipeline/text-image-search">https://towhee.io/tasks/detail/pipeline/text-image-search</a></p>
|
|
<figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">from</span> towhee <span class="keyword">import</span> AutoPipes, AutoConfig</span><br><span class="line"></span><br><span class="line"><span class="comment"># set MilvusInsertConfig for the built-in insert_milvus pipeline</span></span><br><span class="line">insert_conf = AutoConfig.load_config(<span class="string">'insert_milvus'</span>)</span><br><span class="line">insert_conf.collection_name = <span class="string">'text_image_search'</span></span><br><span class="line"></span><br><span class="line">insert_pipe = AutoPipes.pipeline(<span class="string">'insert_milvus'</span>, insert_conf)</span><br><span class="line"></span><br><span class="line"><span class="comment"># generate embedding</span></span><br><span class="line">embedding = image_embedding(<span class="string">'./test1.png'</span>).get()[<span class="number">0</span>]</span><br><span class="line"></span><br><span class="line"><span class="comment"># insert text and embedding into Milvus</span></span><br><span class="line">insert_pipe([<span class="string">'./test1.png'</span>, embedding])</span><br></pre></td></tr></table></figure>
|
|
<p>第三个坑，这并不能跑起来</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"> embedding = insert_pipe.image_embedding('./test.jpg').get()[0]</span><br><span class="line"> ^^^^^^^^^^^^^^^^^^^^^^^^^^^</span><br><span class="line">AttributeError: 'RuntimePipeline' object has no attribute 'image_embedding'</span><br></pre></td></tr></table></figure>
|
|
<p>因为找不到这个 <code>image_embedding</code><br>估计是这个也是包路径变更了，但是并没有什么文档可以找到，之前LlamaIndex还有点更新指南，后来是问了Claude才知道了</p>
|
|
<figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">from</span> towhee <span class="keyword">import</span> AutoPipes, AutoConfig, ops</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment"># set MilvusInsertConfig for the built-in insert_milvus pipeline</span></span><br><span class="line">insert_conf = AutoConfig.load_config(<span class="string">'insert_milvus'</span>)</span><br><span class="line">insert_conf.collection_name = <span class="string">'text_image_search'</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line">insert_pipe = AutoPipes.pipeline(<span class="string">'insert_milvus'</span>, insert_conf)</span><br><span class="line"></span><br><span class="line"><span class="comment"># 创建图像嵌入管道</span></span><br><span class="line">image_embedding_pipe = AutoPipes.pipeline(<span class="string">'image-embedding'</span>)</span><br><span class="line"></span><br><span class="line"><span class="comment"># 生成嵌入</span></span><br><span class="line">embedding = image_embedding_pipe(<span class="string">'./test.jpg'</span>).get()[<span class="number">0</span>]</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment"># insert text and embedding into Milvus</span></span><br><span class="line">insert_pipe([<span class="string">'./test.jpg'</span>, embedding])</span><br></pre></td></tr></table></figure>
|
|
<p>需要从 AutoPipes 中把这个 <code>image-embedding</code> 找出来<br>然后就是上面说到的字段映射问题，默认是先url，再embedding字段，并且主键字段必须是autoId，不然也会缺少默认值，还是感叹下，工程化的东西还是要工程化的质量保证，否则变更都无从知晓，现在人工智能大热，大家都在追风，只是基础的软件还是要稳扎稳打，这样我们工程人员才能把它们更好的用起来，PS：娃真可爱，但真的好累</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>算法</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>算法</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>我是如何走上跑步这条不归路的</title>
|
|
<url>/2020/07/26/%E6%88%91%E6%98%AF%E5%A6%82%E4%BD%95%E8%B5%B0%E4%B8%8A%E8%B7%91%E6%AD%A5%E8%BF%99%E6%9D%A1%E4%B8%8D%E5%BD%92%E8%B7%AF%E7%9A%84/</url>
|
|
<content><![CDATA[<p>这周因为没有准备技术方面的内容加之之前有想分享下我和跑步的一些事情，我从小学开始就是个体育渣，因为体重大非常胖，小学的时候要做仰卧起坐，基本是一个都起不来，然后那时候跑步也是要我命那种，跟另外一个比较胖的同学在跑步队尾苟延残喘，只有立定跳远还行。</p>
|
|
<p>时光飞逝，我在初中高中的时候因为爱打篮球，以为自己体质已经有了质的变化，所以在体育课跑步的时候妄图跟一位体育非常好的同学一起跑，结果跟的快断气了，最终还是确认了自己是个体育渣，特别是到了大学的第一次体测跑一千米，跑完直接吐了，一则是大学太宅不运动，二则的确是底子不好。那么怎么会去跑步了呢，其实也没什么特殊的原因，就是工作以后因为运动得更少，体质差，而且越来越胖，所以就想运动下，加之跑步也是我觉得成本最低的运动了，刚好那时候17 年租的地方附近小区周围的路车不太多，一圈刚好一公里多，就觉得开始跑跑看，其实想想以前觉得一千米是非常远的，学校塑胶跑道才 400 米，一千米要两圈半，太难了，但是后来在这个小区周围跑的时候好像跑了一圈以后还能再跑一点，最后跑了两圈，可把自己牛坏了，我都能跑两千米了，哈哈，这是个什么概念呢，大学里最让我绝望的两项体育相关的内容就是一千米和十二分钟跑，一千米把我跑吐了，十二分钟跑及格五圈半也能让我跑完花一周时间恢复以及提前一周心理压力爆炸，虽然我那时候跑的不快，但是已经能跑两千米了，瞬间让自己自信心爆炸，并且跑完步出完汗的感觉是非常棒的，毕竟吃奶茶鸡排都能心安理得了，谁叫我跑步了呢😄，其实现在回去看，那时候跑得还算快的，因为还比较瘦，现在要跑得那么快心跳就太快了，关于心跳什么的后面说，开始建立起自信心之后，对跑步这件事就开始不那么排斥跟害怕了，毕竟我能跑两千米了，然后试试三公里，哇，也可以了呢，三公里是什么概念呢，我大学里跑过最多的一次是十二分钟跑五圈半还是六圈，也就是两公里多，不到三公里，几乎是生涯最长了，一时间产生了一些我可能是个被埋没的运动天才的错觉，其实细想下也能明白，只是速度足够慢了就能跑多一点，毕竟提测一千米是要跑进四分钟才及格，自己跑的时候一千米跑六分多钟已经算不慢了（对我自己来说），但是即使是这样还是对把跑步坚持下去这件事有了很大的正面激励作用，并且由于那时候上下班骑车，整个体重控制的比较理想，导致一时间误会跑步就能非常快的减肥（其实这是我跑步历程中比较大的误区之一），因为会在跑步前后称下体重，如果跑个五公里（后面可以跑五公里了），可能体重就能降 0.5 千克，但实际上这只是这五公里跑步身体流失的水分，喝杯水就回来了，那时候能控制体重主要是骑车跟跑步一起的作用，并且工作压力相对来讲比较小，没有过劳肥。</p>
|
|
<p>后面其实跑步慢慢变得一个比较习惯的运动了，从三公里，到五公里，到七公里，再到十公里，十公里差不多对我来说是个坎，一直还不能比较轻松的跑十公里，可能近一两年好了一些（原谅我只是跟自己比较，跟那些大神比差得不知道多远），其实对我来说每次都是个突破，因为其实与他人比较没有特别大意义，比较顶尖的差得太远，比较普通的也不行，都会打击自信心，比较比我差的就更没意义了，所以其实能挑战自己，能把自己的上限提高才是最有意义的，这也是我看着朋友圈里的一些大神的速度除了佩服赞叹之外没什么其他的惭愧或者说嫌弃自己的感觉（阿 Q 精神😄）。</p>
|
|
<p>一直感性地觉得，跑步能解压，跑完浑身汗，有种把身体的负能量都排出去的感觉，也把吃太多的罪恶感排解掉了🤦♂️，之前朋友有看一本书，书名差不多叫越跑越接近自己，这个也是我觉得挺准确的一句话，当跑到接近极限了，还想再继续再跑一点，再跑一点就能突破自己上一次的最远记录了，再跑一点就能又一次突破自己了，成人以后，毕业以后，进入社会以后，世事总是难以件件顺遂，磕磕绊绊的往前走，总觉得要崩溃了，但是还是得坚持，再熬一下，再拼一下，可能还是失败，但人生呢，运气好的人和事总是小概率的，唯有面对挫折，还是日拱一卒，来日方长，我们再坚持下，没准下一次，没准再跑一会，就能突破自己，达到新的境界。</p>
|
|
<p>另外个人后期对跑步的一些知识和理解也变得深入一些，比如伤膝盖，其实跑步的确伤膝盖，需要做一些准备和防护，最好的是适合自己的跑鞋和比较好的路（最好的是塑胶跑道了），也要注意热身跟跑后的拉伸（虽然我做的很差），还有就是注意心率，每个人有自己的适宜心率，我这就不冒充科普达人了，可以自行搜索关键字，先说到这吧~</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>运动</category>
|
|
<category>跑步</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>运动</tag>
|
|
<tag>减肥</tag>
|
|
<tag>跑步</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>差点忘了还有spring ai这个包</title>
|
|
<url>/2025/08/16/%E5%B7%AE%E7%82%B9%E5%BF%98%E4%BA%86%E8%BF%98%E6%9C%89springb-ai%E8%BF%99%E4%B8%AA%E5%8C%85/</url>
|
|
<content><![CDATA[<p>之前经常看一些LLM的工具，类似于ollama，chatbox等等，这些都类似于是个独立工具，真的在代码里使用的话可能没那么方便<br>正好看了下spring有个spring-ai的包，可以来使用试下<br>首先我们要用比较新的jdk版本，比如jdk17，然后需要使用springboot比较新的版本，比如3.2.3，但是不建议使用最新稳定版，3.4.x，因为会有bug，别问我怎么知道的。<br>真的是折腾，<br>简单分享下吧<br>需要在pom中增加仓库定义</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="comment"><!-- 仓库定义 --></span></span><br><span class="line"> <span class="tag"><<span class="name">repositories</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">repository</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">id</span>></span>spring-milestones<span class="tag"></<span class="name">id</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">name</span>></span>Spring Milestones<span class="tag"></<span class="name">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">url</span>></span>https://repo.spring.io/milestone<span class="tag"></<span class="name">url</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">snapshots</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">enabled</span>></span>false<span class="tag"></<span class="name">enabled</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">snapshots</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">repository</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">repository</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">id</span>></span>spring-snapshots<span class="tag"></<span class="name">id</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">name</span>></span>Spring Snapshots<span class="tag"></<span class="name">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">url</span>></span>https://repo.spring.io/snapshot<span class="tag"></<span class="name">url</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">releases</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">enabled</span>></span>false<span class="tag"></<span class="name">enabled</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">releases</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">repository</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">repositories</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>然后增加依赖</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">dependencyManagement</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">dependencies</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.ai<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-ai-bom<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>1.0.0-M5<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">type</span>></span>pom<span class="tag"></<span class="name">type</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">scope</span>></span>import<span class="tag"></<span class="name">scope</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependencies</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependencyManagement</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>引入主要的几个包</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.boot<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-boot-starter<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.boot<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-boot-starter-web<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.ai<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-ai-openai-spring-boot-starter<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>代码也很简单，但是不确定是api限流太严重还是啥，响应慢的离谱</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@RestController</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">ChatController</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> ChatClient chatClient;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">ChatController</span><span class="params">(ChatClient.Builder builder)</span> {</span><br><span class="line"> <span class="built_in">this</span>.chatClient = builder.defaultSystem(<span class="string">"你是个资深服务端架构师"</span>).build();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@GetMapping("/demo/{message}")</span></span><br><span class="line"> <span class="keyword">public</span> String <span class="title function_">demo</span><span class="params">(<span class="meta">@PathVariable("message")</span> String message)</span> {</span><br><span class="line"></span><br><span class="line"> message = <span class="string">"怎么学习java"</span>;</span><br><span class="line"> <span class="type">String</span> <span class="variable">response</span> <span class="operator">=</span> chatClient.prompt()</span><br><span class="line"> .user(message)</span><br><span class="line"> .call().content();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> response;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>对的，甚至把prompt写死了<br><img data-src="https://img.nicksxs.me/lajihuoshan.png"><br>这里经过了好几个包版本的调整<br>原先比较早的0.8的不支持接口的自定义路径，因为要用火山的接口，它的接口跟openai是不一样的，后面有个v3啥的<br>所以需要增加最下面一行</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">spring.ai.openai.base-url=https://ark.cn-beijing.volces.com/api/v3</span><br><span class="line">spring.ai.openai.chat.options.model=deepseek-r1-250120</span><br><span class="line">spring.ai.openai.chat.completions-path=/chat/completions</span><br></pre></td></tr></table></figure>
|
|
<p>但是可以从上面的图里看出来，响应非常慢，简直不能忍，有openai 权限的可以帮忙试下究竟是啥原因，抓包啥的也不太方便</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>折腾记-给玩客云装上casaos</title>
|
|
<url>/2024/04/28/%E6%8A%98%E8%85%BE%E8%AE%B0-%E7%BB%99%E7%8E%A9%E5%AE%A2%E4%BA%91%E8%A3%85%E4%B8%8Acasaos/</url>
|
|
<content><![CDATA[<h1 id="玩客云-casaos"><a href="#玩客云-casaos" class="headerlink" title="玩客云 casaos"></a>玩客云 casaos</h1><p>玩客云这样的小机器现在也有一些自己的生态了，感谢开源大佬们的贡献，这个casaos就是个颜值颇高的轻nas方案，casaos虽然叫做os，但其实一套轻nas组件，并且有一个好看的webui，然后通过docker来作为内部的软件市场安装形式，我们沿着上一篇接下来尝试下安装轻度试用下这个casaos，首先安装</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"></span><br><span class="line">curl -fsSL https://get.casaos.io | sudo bash</span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
|
|
<p>安装会提示空间不够大，可以先忽略，如果想更保险的话就可以通过加一张SD卡或者USB插外接硬盘<br><img data-src="https://img.nicksxs.me/blog/QHH3xp.png"></p>
|
|
<p>安装中主要是执行了casaos的一些安装shell，以及启动rclone服务，完成后给出了登录地址，默认是80端口，可以在后面UI界面里更改端口</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/uXefRv.png"><br>也可以设置自动挂载usb</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/yHWAhW.png"></p>
|
|
<p>安装完以后打开地址</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/gn3PUp.png"></p>
|
|
<p>创建账户</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/87yM37.png"></p>
|
|
<p>进入系统后的UI长这样</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/mJBFXL.png"><br>还是挺赏心悦目的，默认会展示系统的状态，cpu，内存和存储网络这些，预装了syncthing作为同步工具，还有个App Store<br><img data-src="https://img.nicksxs.me/blog/jqKgya.png"><br>有挺多应用在商店</p>
|
|
<p>只是这里就会发现个问题，只是玩客云是32为的ArmV7系统</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/TWeezZ.png"></p>
|
|
<p>有很多组件不支持，包括jellyfin这些，不过它提供了自定义安装功能，可以自定义安装docker镜像</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/XVpncH.png"></p>
|
|
<p>不过这里有个小bug，拷贝镜像名进入第一个输入框以后会提示不能为空，这边需要带上tag一起复制进去</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/JKJiN7.png"></p>
|
|
<p>第二个问题是我安装的0.4.8是没办法安装latest以外的tag的，不然会提示 store app not found</p>
|
|
<p>在dockerhub上是有armv7版本的jellyfin的，只是没有latest这个tag的，如果实在想体验下就考虑安装casaos的0.4.7或更早先版本的或者可以自己在Armbian里拉取镜像再安装下Portainer来管理docker，不过玩客云的性能是真的很羸弱，所以不要抱什么期望，低负载的任务可以跑跑，jellyfin显然不是很合适的，只是作为一个示例</p>
|
|
<p>逻辑复杂负载比较高的就算了，考虑N1或者其他性能更好的机器</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>折腾</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>折腾</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>怎么给wsl占用的硬盘空间做下瘦身</title>
|
|
<url>/2025/09/27/%E6%80%8E%E4%B9%88%E7%BB%99wsl%E5%8D%A0%E7%94%A8%E7%9A%84%E7%A1%AC%E7%9B%98%E7%A9%BA%E9%97%B4%E5%81%9A%E4%B8%8B%E7%98%A6%E8%BA%AB/</url>
|
|
<content><![CDATA[<p>wsl出来这些年感觉在windows里使用linux，或者ubuntu真的是方便了很多，不用搞虚拟机这些<br>但是也有一些明显的问题，首先就是默认它是在C盘中安装，长久使用占用的空间就会越来越大，而且对于一些临时使用占用空间后，即使删除了文件，windows并不会对已经扩容的虚拟磁盘文件进行缩容<br>所以就介绍下怎么手动来给它释放空间<br>首先可以看下安装的系统有哪些</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">wsl -l -v</span><br></pre></td></tr></table></figure>
|
|
<p>比如我这就是一个ubuntu 20.04</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"> NAME STATE VERSION</span><br><span class="line">* Ubuntu-20.04 Running 2</span><br></pre></td></tr></table></figure>
|
|
<p>然后我们需要寻找下虚拟磁盘的所在位置<br>我的是在<br><code>C:\Users\Administrator\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc\LocalState\ext4.vhdx</code><br>可以按文件名 <code>ext4.vhdx</code> 进行查找<br>当然在操作之前我们还是要做好备份工作，并且还有特殊功效</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_"># </span><span class="language-bash">首先进行关机</span></span><br><span class="line">wsl --shutdown</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">然后做打包备份到制定路径，放心这里打包的大小就是实际大小</span></span><br><span class="line">wsl --export Ubuntu-20.04 X:\Ubuntu-20.04.tar</span><br></pre></td></tr></table></figure>
|
|
<p>然后我们运行 <code>diskpart</code> 命令，这是windows用来管理磁盘文件的工具<br>运行后会进入到 <code>diskpart</code> 工具的命令状态下</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">select vdisk file="C:\Users\Administrator\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc\LocalState\ext4.vhdx"</span><br></pre></td></tr></table></figure>
|
|
<p>接着我们再运行<code>select vdisk</code>命令来选中这个文件<br>选中以后会提示<br><code>DiskPart 已成功选择虚拟磁盘文件。</code><br>接着运行压缩命令</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">compact vdisk</span><br></pre></td></tr></table></figure>
|
|
<p>运行时会有个压缩进度展示，压缩完成后会显示以下内容</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"> 100 百分比已完成</span><br><span class="line"></span><br><span class="line">DiskPart 已成功压缩虚拟磁盘文件。</span><br></pre></td></tr></table></figure>
|
|
<p>最终我们运行</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">detach vdisk</span><br></pre></td></tr></table></figure>
|
|
<p>分离命令就可以了<br>这样子我们的wsl因为扩容后无法自动收回的空间就被我们压缩出来了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>wsl</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>wsl</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>搬运两个 StackOverflow 上的 Mysql 编码相关的问题解答</title>
|
|
<url>/2022/01/16/%E6%90%AC%E8%BF%90%E4%B8%A4%E4%B8%AA-StackOverflow-%E4%B8%8A%E7%9A%84-Mysql-%E7%BC%96%E7%A0%81%E7%9B%B8%E5%85%B3%E7%9A%84%E9%97%AE%E9%A2%98%E8%A7%A3%E7%AD%94/</url>
|
|
<content><![CDATA[<h1 id="Mysql-字符编码和排序规则"><a href="#Mysql-字符编码和排序规则" class="headerlink" title="Mysql 字符编码和排序规则"></a>Mysql 字符编码和排序规则</h1><p>这个一直是属于一知半解的状态，知道 utf8 跟 utf8mb4 的区别主要是能不能支持 emoji，但是具体后面配置的排序规则是用来干嘛，或者有什么区别，应该使用哪个，所以在 stackoverflow 上找了下，有两个比较不错的解答，就搬过来并且配合机翻做了点修改</p>
|
|
<h3 id="原文"><a href="#原文" class="headerlink" title="原文"></a><a href="https://stackoverflow.com/questions/766809/whats-the-difference-between-utf8-general-ci-and-utf8-unicode-ci">原文</a></h3><p>For those people still arriving at this question in 2020 or later, there are newer options that may be better than both of these. For example, <code>utf8mb4_0900_ai_ci</code>.</p>
|
|
<p>All these collations are for the UTF-8 character encoding. The differences are in how text is sorted and compared.</p>
|
|
<p><code>_unicode_ci</code> and <code>_general_ci</code> are two different sets of rules for sorting and comparing text according to the way we expect. Newer versions of MySQL introduce new sets of rules, too, such as <code>_0900_ai_ci</code> for equivalent rules based on Unicode 9.0 - and with no equivalent <code>_general_ci</code> variant. People reading this now should probably use one of these newer collations instead of either <code>_unicode_ci</code> or <code>_general_ci</code>. The description of those older collations below is provided for interest only.</p>
|
|
<p>MySQL is currently transitioning away from an older, flawed UTF-8 implementation. For now, you need to use utf8mb4 instead of utf8 for the character encoding part, to ensure you are getting the fixed version. The flawed version remains for backward compatibility, though it is being deprecated.</p>
|
|
<p>Key differences</p>
|
|
<p><code>utf8mb4_unicode_ci</code> is based on the official Unicode rules for universal sorting and comparison, which sorts accurately in a wide range of languages.</p>
|
|
<p><code>utf8mb4_general_ci</code> is a simplified set of sorting rules which aims to do as well as it can while taking many short-cuts designed to improve speed. It does not follow the Unicode rules and will result in undesirable sorting or comparison in some situations, such as when using particular languages or characters.</p>
|
|
<p>On modern servers, this performance boost will be all but negligible. It was devised in a time when servers had a tiny fraction of the CPU performance of today’s computers.</p>
|
|
<p>Benefits of <code>utf8mb4_unicode_ci</code> over <code>utf8mb4_general_ci</code></p>
|
|
<p>utf8mb4_unicode_ci, which uses the Unicode rules for sorting and comparison, employs a fairly complex algorithm for correct sorting in a wide range of languages and when using a wide range of special characters. These rules need to take into account language-specific conventions; not everybody sorts their characters in what we would call ‘alphabetical order’.</p>
|
|
<p>As far as Latin (ie “European”) languages go, there is not much difference between the Unicode sorting and the simplified <code>utf8mb4_general_ci</code>sorting in MySQL, but there are still a few differences:</p>
|
|
<p>For examples, the Unicode collation sorts “ß” like “ss”, and “Œ” like “OE” as people using those characters would normally want, whereas <code>utf8mb4_general_ci</code>sorts them as single characters (presumably like “s” and “e” respectively).</p>
|
|
<p>Some Unicode characters are defined as ignorable, which means they shouldn’t count toward the sort order and the comparison should move on to the next character instead. <code>utf8mb4_unicode_ci</code>handles these properly.</p>
|
|
<p>In non-latin languages, such as Asian languages or languages with different alphabets, there may be a lot more differences between Unicode sorting and the simplified <code>utf8mb4_general_ci</code>sorting. The suitability of <code>utf8mb4_general_ci</code>will depend heavily on the language used. For some languages, it’ll be quite inadequate.</p>
|
|
<p>What should you use?</p>
|
|
<p>There is almost certainly no reason to use <code>utf8mb4_general_ci</code>anymore, as we have left behind the point where CPU speed is low enough that the performance difference would be important. Your database will almost certainly be limited by other bottlenecks than this.</p>
|
|
<p>In the past, some people recommended to use <code>utf8mb4_general_ci</code>except when accurate sorting was going to be important enough to justify the performance cost. Today, that performance cost has all but disappeared, and developers are treating internationalization more seriously.</p>
|
|
<p>There’s an argument to be made that if speed is more important to you than accuracy, you may as well not do any sorting at all. It’s trivial to make an algorithm faster if you do not need it to be accurate. So, <code>utf8mb4_general_ci</code>is a compromise that’s probably not needed for speed reasons and probably also not suitable for accuracy reasons.</p>
|
|
<p>One other thing I’ll add is that even if you know your application only supports the English language, it may still need to deal with people’s names, which can often contain characters used in other languages in which it is just as important to sort correctly. Using the Unicode rules for everything helps add peace of mind that the very smart Unicode people have worked very hard to make sorting work properly.</p>
|
|
<p>What the parts mean</p>
|
|
<p>Firstly, ci is for case-insensitive sorting and comparison. This means it’s suitable for textual data, and case is not important. The other types of collation are cs (case-sensitive) for textual data where case is important, and bin, for where the encoding needs to match, bit for bit, which is suitable for fields which are really encoded binary data (including, for example, Base64). Case-sensitive sorting leads to some weird results and case-sensitive comparison can result in duplicate values differing only in letter case, so case-sensitive collations are falling out of favor for textual data - if case is significant to you, then otherwise ignorable punctuation and so on is probably also significant, and a binary collation might be more appropriate.</p>
|
|
<p>Next, unicode or general refers to the specific sorting and comparison rules - in particular, the way text is normalized or compared. There are many different sets of rules for the utf8mb4 character encoding, with unicode and general being two that attempt to work well in all possible languages rather than one specific one. The differences between these two sets of rules are the subject of this answer. Note that unicode uses rules from Unicode 4.0. Recent versions of MySQL add the rulesets unicode_520 using rules from Unicode 5.2, and 0900 (dropping the “unicode_” part) using rules from Unicode 9.0.</p>
|
|
<p>And lastly, utf8mb4 is of course the character encoding used internally. In this answer I’m talking only about Unicode based encodings.</p>
|
|
<h3 id="翻译"><a href="#翻译" class="headerlink" title="翻译"></a>翻译</h3><p>对于那些在 2020 年或之后仍会遇到这个问题的人，有可能比这两个更好的新选项。例如，<code>utf8mb4_0900_ai_ci</code>。</p>
|
|
<p>所有这些排序规则都用于 UTF-8 字符编码。不同之处在于文本的排序和比较方式。</p>
|
|
<p><code>_unicode_ci</code>和 <code>_general_ci</code>是两组不同的规则，用于按照我们期望的方式对文本进行排序和比较。较新版本的 MySQL 也引入了新的规则集，例如 <code>_0900_ai_ci</code>用于基于 Unicode 9.0 的等效规则 - 并且没有等效的 <code>_general_ci</code>变体。现在阅读本文的人可能应该使用这些较新的排序规则之一，而不是 <code>_unicode_ci</code>或 <code>_general_ci</code>。下面对那些较旧的排序规则的描述仅供参考。</p>
|
|
<p>MySQL 目前正在从旧的、有缺陷的 UTF-8 实现过渡。现在，您需要使用 <code>utf8mb4</code> 而不是 <code>utf8</code>作为字符编码部分，以确保您获得的是固定版本。有缺陷的版本仍然是为了向后兼容，尽管它已被弃用。</p>
|
|
<p>主要区别</p>
|
|
<p><code>utf8mb4_unicode_ci</code>基于官方 Unicode 规则进行通用排序和比较，可在多种语言中准确排序。</p>
|
|
<p><code>utf8mb4_general_ci</code>是一组简化的排序规则，旨在尽其所能，同时采用许多旨在提高速度的捷径。它不遵循 Unicode 规则，并且在某些情况下会导致不希望的排序或比较，例如在使用特定语言或字符时。</p>
|
|
<p>在现代服务器上，这种性能提升几乎可以忽略不计。它是在服务器的 CPU 性能只有当今计算机的一小部分时设计的。</p>
|
|
<p><code>utf8mb4_unicode_ci</code> 相对于 <code>utf8mb4_general_ci</code>的优势</p>
|
|
<p><code>utf8mb4_unicode_ci</code>使用 Unicode 规则进行排序和比较，采用相当复杂的算法在多种语言中以及在使用多种特殊字符时进行正确排序。这些规则需要考虑特定语言的约定；不是每个人都按照我们所说的“字母顺序”对他们的字符进行排序。</p>
|
|
<p>就拉丁语（即“欧洲”）语言而言，Unicode 排序和 MySQL 中简化的 <code>utf8mb4_general_ci</code>排序没有太大区别，但仍有一些区别：</p>
|
|
<p>例如，Unicode 排序规则将“ß”排序为“ss”，将“Œ”排序为“OE”，因为使用这些字符的人通常需要这些字符，而 <code>utf8mb4_general_ci</code>将它们排序为单个字符（大概分别像“s”和“e” ）。</p>
|
|
<p>一些 Unicode 字符被定义为可忽略，这意味着它们不应该计入排序顺序，并且比较应该转到下一个字符。 <code>utf8mb4_unicode_ci</code>正确处理这些。</p>
|
|
<p>在非拉丁语言中，例如亚洲语言或具有不同字母的语言，Unicode 排序和简化的 <code>utf8mb4_general_ci</code>排序之间可能存在更多差异。 <code>utf8mb4_general_ci</code>的适用性在很大程度上取决于所使用的语言。对于某些语言，这将是非常不充分的。</p>
|
|
<p>你应该用什么？</p>
|
|
<p>几乎可以肯定没有理由再使用 <code>utf8mb4_general_ci</code>，因为我们已经将 CPU 速度低到会严重影响性能表现的时代远抛在脑后了。您的数据库几乎肯定会受到除此之外的其他瓶颈的限制。</p>
|
|
<p>过去，有些人建议使用 <code>utf8mb4_general_ci</code>，除非准确排序足够重要以证明性能成本是合理的。如今，这种性能成本几乎消失了，开发人员正在更加认真地对待国际化。</p>
|
|
<p>有一个论点是，如果速度对您来说比准确性更重要，那么您可能根本不进行任何排序。如果您不需要准确的算法，那么使算法更快是微不足道的。因此，<code>utf8mb4_general_ci</code>是一种折衷方案，出于速度原因可能不需要，也可能出于准确性原因也不适合。</p>
|
|
<p>我要补充的另一件事是，即使您知道您的应用程序仅支持英语，它可能仍需要处理人名，这些人名通常包含其他语言中使用的字符，在这些语言中正确排序同样重要.对所有事情都使用 Unicode 规则有助于让您更加安心，因为非常聪明的 Unicode 人员已经非常努力地工作以使排序正常工作。</p>
|
|
<p>其余各个部分是什么意思</p>
|
|
<p>首先， <code>ci</code> 用于不区分大小写的排序和比较。这意味着它适用于文本数据，大小写并不重要。其他类型的排序规则是 <code>cs</code>（区分大小写），用于区分大小写的文本数据，以及 <code>bin</code>，用于编码需要匹配的地方，逐位匹配，适用于真正编码二进制数据的字段（包括，用于例如，Base64）。区分大小写的排序会导致一些奇怪的结果，区分大小写的比较可能会导致重复值仅在字母大小写上有所不同，因此区分大小写的排序规则对文本数据不受欢迎 - 如果大小写对您很重要，那么标点符号就可以忽略等等可能也很重要，二进制排序规则可能更合适。</p>
|
|
<p>接下来，unicode 或general 指的是具体的排序和比较规则——特别是文本被规范化或比较的方式。 utf8mb4 字符编码有许多不同的规则集，其中 unicode 和 general 是两种，它们试图在所有可能的语言中都很好地工作，而不是在一种特定的语言中。这两组规则之间的差异是此答案的主题。请注意，unicode 使用 Unicode 4.0 中的规则。 MySQL 的最新版本使用 Unicode 5.2 的规则添加规则集 unicode_520，使用 Unicode 9.0 的规则添加 0900（删除“unicode_”部分）。</p>
|
|
<p>最后，utf8mb4 当然是内部使用的字符编码。在这个答案中，我只谈论基于 Unicode 的编码。</p>
|
|
<h1 id="utf8-和-utf8mb4-编码有什么区别"><a href="#utf8-和-utf8mb4-编码有什么区别" class="headerlink" title="utf8 和 utf8mb4 编码有什么区别"></a>utf8 和 utf8mb4 编码有什么区别</h1><h3 id="原文-1"><a href="#原文-1" class="headerlink" title="原文"></a><a href="https://stackoverflow.com/questions/30074492/what-is-the-difference-between-utf8mb4-and-utf8-charsets-in-mysql">原文</a></h3><p><a href="https://en.wikipedia.org/wiki/UTF-8">UTF-8</a>is a variable-length encoding. In the case of UTF-8, this means that storing one code point requires one to four bytes. However, MySQL’s encoding called “utf8” (alias of “utf8mb3”) only stores a maximum of three bytes per code point.</p>
|
|
<p>So the character set “utf8”/“utf8mb3” cannot store all Unicode code points: it only supports the range 0x000 to 0xFFFF, which is called the “<a href="http://en.wikipedia.org/wiki/Plane_%28Unicode%29#Basic_Multilingual_Plane">Basic Multilingual Plane</a>“. See also <a href="http://en.wikipedia.org/wiki/Comparison_of_Unicode_encodings#In_detail">Comparison of Unicode encodings</a>.</p>
|
|
<p>This is what (a previous version of the same page at)<a href="https://dev.mysql.com/doc/refman/5.5/en/charset-unicode-utf8mb4.html">the MySQL documentation</a>has to say about it:</p>
|
|
<blockquote>
|
|
<p>The character set named utf8[/utf8mb3] uses a maximum of three bytes per character and contains only BMP characters. As of MySQL 5.5.3, the utf8mb4 character set uses a maximum of four bytes per character supports supplemental characters:</p>
|
|
<ul>
|
|
<li>For a BMP character, utf8[/utf8mb3] and utf8mb4 have identical storage characteristics: same code values, same encoding, same length.</li>
|
|
<li>For a supplementary character, <strong>utf8[/utf8mb3] cannot store the character at all</strong>, while utf8mb4 requires four bytes to store it. Since utf8[/utf8mb3] cannot store the character at all, you do not have any supplementary characters in utf8[/utf8mb3] columns and you need not worry about converting characters or losing data when upgrading utf8[/utf8mb3] data from older versions of MySQL.</li>
|
|
</ul>
|
|
</blockquote>
|
|
<p>So if you want your column to support storing characters lying outside the BMP (and you usually want to), such as <a href="https://en.wikipedia.org/wiki/Emoji">emoji</a>, use “utf8mb4”. See also <a href="https://stackoverflow.com/questions/5567249/what-are-the-most-common-non-bmp-unicode-characters-in-actual-use">What are the most common non-BMP Unicode characters in actual use?</a>.</p>
|
|
<h3 id="译文"><a href="#译文" class="headerlink" title="译文"></a>译文</h3><p><a href="https://en.wikipedia.org/wiki/UTF-8">UTF-8</a> 是一种可变长度编码。对于 UTF-8，这意味着存储一个代码点需要一到四个字节。但是，MySQL 的编码称为“utf8”（“utf8mb3”的别名）每个代码点最多只能存储三个字节。</p>
|
|
<p>所以字符集“utf8”/“utf8mb3”不能存储所有的Unicode码位：它只支持0x000到0xFFFF的范围，被称为“<a href="http://en.wikipedia.org/wiki/Plane_%28Unicode%29#Basic_Multilingual_Plane">基本多语言平面</a>”。另请参阅 <a href="http://en.wikipedia.org/wiki/Comparison_of_Unicode_encodings#In_detail">Unicode 编码比较</a>。</p>
|
|
<p>这就是（同一页面的先前版本）<a href="https://dev.mysql.com/doc/refman/5.5/en/charset-unicode-utf8mb4.html">MySQL 文档</a> 不得不说的：</p>
|
|
<blockquote>
|
|
<p>名为 utf8[/utf8mb3] 的字符集每个字符最多使用三个字节，并且仅包含 BMP 字符。从 MySQL 5.5.3 开始，utf8mb4 字符集每个字符最多使用四个字节，支持补充字符：</p>
|
|
<ul>
|
|
<li>对于 BMP 字符，utf8[/utf8mb3] 和 utf8mb4 具有相同的存储特性：相同的代码值、相同的编码、相同的长度。</li>
|
|
<li>对于补充字符，<strong>utf8[/utf8mb3] 根本无法存储该字符</strong>，而 utf8mb4 需要四个字节来存储它。由于 utf8[/utf8mb3] 根本无法存储字符，因此您在 utf8[/utf8mb3] 列中没有任何补充字符，您不必担心从旧版本升级 utf8[/utf8mb3] 数据时转换字符或丢失数据mysql。</li>
|
|
</ul>
|
|
</blockquote>
|
|
<p>因此，如果您希望您的列支持存储位于 BMP 之外的字符（并且您通常希望这样做），例如 <a href="https://en.wikipedia.org/wiki/Emoji">emoji</a>，请使用“utf8mb4”。另请参阅</p>
|
|
<p><a href="https://stackoverflow.com/questions/5567249/what-are-the-most-common-non-bmp-unicode-characters-in-actual-use">实际使用中最常见的非 BMP Unicode 字符是什么？</a> 。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Mysql</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>mysql</tag>
|
|
<tag>字符集</tag>
|
|
<tag>编码</tag>
|
|
<tag>utf8</tag>
|
|
<tag>utf8mb4</tag>
|
|
<tag>utf8mb4_0900_ai_ci</tag>
|
|
<tag>utf8mb4_unicode_ci</tag>
|
|
<tag>utf8mb4_general_ci</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>折腾记-给玩客云刷上Armbian</title>
|
|
<url>/2024/03/31/%E6%8A%98%E8%85%BE%E8%AE%B0-%E7%BB%99%E7%8E%A9%E5%AE%A2%E4%BA%91%E5%88%B7%E4%B8%8AArmbian/</url>
|
|
<content><![CDATA[<p>之前买来玩客云作为轻nas使用，但是由于只能外接硬盘，并且是usb2.0的接口，使用体验不是特别好，并且前期作为老母鸡还能兑换视频会员这种，后面取消了就不怎么使用了，前阵子玩客云停止提供服务了，所以就想可以折腾下，比如Armbian这样的，可以作为轻量服务器使用，功耗也比较低，跑一些简单的docker还是可以的，<br><img data-src="https://img.nicksxs.me/blog/JzeFde.png"><br>配置是这样的，存储小了点，其他就是一个普通小服务器的配置了，usb2.0有点弱，但是现在有一些是正常退役，还有很多是PCDN被禁了宽带以后咸鱼出售的，带上电源跟网线才30左右，还是比较有性价比的<br>首先机器有几个版本，主要是1.0跟1.3，要注意1.3版本的主板上有个混淆点，会有一个1.2的版本，指的是具体存储或者网卡等组件的版本，玩客云版本是较大的 V1.3，而1.0则是比较小的字体，而且据网上的资料，1.0版本是有个MAC地址的贴条<br><img data-src="https://img.nicksxs.me/blog/%E5%BE%AE%E4%BF%A1%E5%9B%BE%E7%89%87%E7%BC%96%E8%BE%91_20240331182436.jpg"><br>可以看上图的标示，箭头指向的是短接点，这是1.0版本的，1.3版本是找不到这个短接点，而是在反面（有S805芯片和存储的一面）<br>新版手边没有，借用一张恩山的图<br><img data-src="https://img.nicksxs.me/blog/GiOBE9.png"><br>这个在最开始我刷第一个的时候比较困扰我，因为不确定版本，网上又有很多都是只截一小块区域的图，也不知道相对位置<br>首先如果是从闲鱼买来的已经刷成PCDN，某心云的需要通过短接刷底包之后再刷入系统，可以先用吹风机热风吹一会后盖，然后用一字螺丝刀从SD卡槽把外层的一片壳撬开，然后通过十字螺丝刀拧开之后就可以把主板拿出来<br>然后需要镊子或者其他铁丝，能够短接的就行，外加一个双公头USB线<br>刷机工具包可以用我这个分享的</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">https://www.alipan.com/s/sd4Wi6TDzeP</span><br><span class="line">提取码: 84or</span><br></pre></td></tr></table></figure>
|
|
<p>第一步打开USB Burning Tool, 要短接刷入update.img<br><img data-src="https://img.nicksxs.me/blog/%E5%BE%AE%E4%BF%A1%E5%9B%BE%E7%89%87_20240331190655.png"><br>通过文件导入网盘里的 update.img，然后先用双公头USB线连接电脑的USB口和玩客云<strong>靠近HDMI口</strong>的USB口，如果方便最好接到电脑后置的USB口，防止不稳定刷机终端，接下来就需要短接了，这里的姿势要先短接好，保持稳定，然后插上电源线，这里如果连接成功就会在软件左上会显示一栏连接成功，如果电脑连接音响也会有叮咚这样的声音，接下去就可以点开始进行刷入底包了，这里最大的困难点就是保持短接稳定后插电源，并且继续保持稳定刷入底包，底包刷入会比较快，等到刷机进度条走完刷成功后就点停止，然后关闭软件，拔掉玩客云电源，拔掉USB<br>第二步是刷入Armbian系统，根据一些教程有些说需要先刷入5.8，我这边是直接刷入了5.9的系统，也就多一步，准备一个U盘，16G或者8G都行，先用Etcher把5.8系统刷入U盘，刷入完成后再关闭软件，推出U盘，将U盘插入玩客云上靠近网口的USB口，然后再通电，指示灯会蓝色-> 绿色 -> 蓝紫闪烁 -> 蓝色，这样就是刷机成功了，会需要一点时间，刷机完成后再把5.9的的刷入U盘，再继续刚才的步骤，此时需要把网线插入玩客云，等待正常亮灯显示后，在路由器上可以找到一个aml-s812的设备，在通过ssh连接，账号是root，密码是1234，进入系统后通过以下命令就可以刷入emmc了</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">cd /boot/install sudo ./install.sh</span><br></pre></td></tr></table></figure>
|
|
<p>完成后会自动断开，拔掉U盘后再用同样方法连接就可以了，后续怎么使用就是Linux相关的了，后面也可以讲下docker的操作，但要注意架构，这个是arm架构32位的</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>折腾</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>折腾</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>是何原因竟让两人深夜奔袭十公里</title>
|
|
<url>/2022/06/05/%E6%98%AF%E4%BD%95%E5%8E%9F%E5%9B%A0%E7%AB%9F%E8%AE%A9%E4%B8%A4%E4%BA%BA%E6%B7%B1%E5%A4%9C%E5%A5%94%E8%A2%AD%E5%8D%81%E5%85%AC%E9%87%8C/</url>
|
|
<content><![CDATA[<p>偶尔来个标题党，不过也是一次比较神奇的经历<br>上周五下班后跟 LD 约好去吃牛蛙，某个朋友好像对这类都不太能接受，我以前小时候也不常吃，但是这类其实都是口味比较重，没有那种肉本身的腥味，而且肉质比较特殊，吃过几次以后就有点爱上了，这次刚好是 LD 买的新店开业券，比较优惠（我们俩都是有点勤俭持家的，想着小电驴还有三格电，这家店又有点远，骑车单趟大概要 10 公里左右，有点担心，LD 说应该可以的，就一起骑了过去（跟她轮换着骑电驴和共享单车），结果大概离吃牛蛙的店还有一辆公里的时候，电量就报警了，只有最后一个红色的了，一共是五格，最后一格是红色的，提示我们该充电了，这样子是真的有点慌了，之前开了几个月都是还有一两格电的时候就充电了，没有试验过究竟这最后一格电能开多远，总之先到了再说。<br>这家牛蛙没想到还挺热闹的，我们到那已经快八点了，还有十几个排队的，有个人还想插队（向来是不惯着这种，一边去），旁边刚好是有些商店就逛了下，就跟常规的商业中心差不多，开业的比较早也算是这一边比较核心的商业综合体了，各种品牌都有，而且还有彩票售卖点的，只是不太理解现在的彩票都是兑图案的，而且要 10 块钱一张，我的概念里还是以前 2 块钱一张的双色球，偶尔能中个五块十块的。排队还剩四五个的时候我们就去门口坐着等了，又等了大概二十分钟才排到我们，靠近我们等的里面的位置，好像好几个小女生在那还叫了外卖奶茶，然后各种拍照，小朋友的生活还是丰富多彩的，我们到了就点了蒜蓉的，没有点传说中紫苏的，菜单上画了 N 个🌶，LD 还是想体验下说下次人多点可以试试，我们俩吃怕太辣了吃不消，口味还是不错的，这家貌似是 LD 闺蜜推荐的，口碑有保证。两个人光吃一个蛙锅就差不多了，本来还想再点个其他的，后面实在吃不下了就没点，吃完还是惯例点了个奶茶，不过是真的不好找，太大了。<br>本来是就回个家的事了，结果就因为前面铺垫的小电驴已经只有一格电了，标题的深夜奔袭十公里就出现了，这个电驴估计续航也虚标挺严重的，电量也是这样，骑的时候显示只有一格电，关掉再开起来又有三格，然后我们回去骑了没一公里就没电了，这下是真的完球了，觉得车子也比较新，直接停外面也不放心，就开始了深夜的十公里推电驴奔袭，LD 看我太累还帮我中间推了一段，虽然是跑过十公里的，但是推着个没电的电驴，还是着实不容易的，LD 也是陪我推着车走，中间好几次说我们把电驴停着打车回去，把电池带回去充满了明天再过来骑车，可能是心态已经转变了，这应该算是一次很特殊的体验，从我们吃完出来大概十点，到最后我们推到小区，大概是过了两个小时的样子，说句深夜也不太过分，把这次这么推车看成了一种意志力的考验，很多事情也都是怕坚持，或者说怕不能坚持，想走得远，没有持续的努力坚持肯定是不行的，所以还是坚持着把车推回来（好吧，我其实主要是怕车被偷，毕竟刚来杭州上学没多久就被偷了自行车留下了阴影），中间感谢 LD，跟我轮着推了一段路，有些下坡的时候还在那坐着用脚蹬一下，离家里大概还有一公里的时候，有个骑电瓶车的大叔还停下来问我们是车破了还是没电了，应该是出于好意吧，最后快到的时候真的非常渴，买了2.5 升的水被我一口气喝了大半瓶，奶茶已经不能起到解渴的作用了，本来以为这样能消耗很多，结果第二天一称还重了，（我的称一定有问题 233</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>折腾记-讲一下iptables的一个小问题</title>
|
|
<url>/2024/05/05/%E6%8A%98%E8%85%BE%E8%AE%B0-%E8%AE%B2%E4%B8%80%E4%B8%8Biptables%E7%9A%84%E4%B8%80%E4%B8%AA%E5%B0%8F%E9%97%AE%E9%A2%98/</url>
|
|
<content><![CDATA[<p>很早之前就有用过iptables，那时候是早期版本的Ubuntu系统，配置防火墙的时候，是挺复杂的，就是在reject之前添加accept规则，并且当时很多网上资料都是在后续添加就行，这个其实在当时是有问题的，所以比较印象深刻。<br>这次碰到的问题是对iptables的概念不熟悉有关，因为iptables的逻辑其实也有一定问题我觉得，比如最基本的命令，<code>iptables -L</code><br>这里help显示的是会展示所有chain的rules<br><img data-src="https://img.nicksxs.me/blog/WzwlYP.png"><br>但是这样完全把table概念给忽略了，如果不注明理论上应该把所有table的展示出来，或者把包含这个参数并且是默认给出filter这个table的写清楚<br>所以这次要补一下这部分概念<br>iptables里首先就是有四种table，分别是 Filter, NAT, Mangle, Raw四种内建表<br>并且层次结构是 iptables -> Tables -> Chains -> Rules.<br>filter:<br>This is the default table (if no -t option is passed). It contains the built-in chains INPUT (for packets destined to local sockets), FORWARD (for packets being routed through the box), and OUTPUT (for locally-generated packets).<br>包含 INPUT，FORWORD和OUPUT三个chain，默认就是展示这个，所以就会导致用<code>iptables -L</code> 显示不了mangle table的规则<br>nat:<br>This table is consulted when a packet that creates a new connection is encountered. It consists of three built-ins: PREROUTING (for altering packets as soon as they come in), OUTPUT (for altering locally-generated packets before routing), and POSTROUTING (for altering packets as they are about to go out).<br>nat包含 PREROUTING，OUTPUT，POSTROUTING三个chain<br>mangle:<br>This table is used for specialized packet alteration. Until kernel 2.4.17 it had two built-in chains: PREROUTING (for altering incoming packets before routing) and OUTPUT (for altering locally-generated packets before routing). Since kernel 2.4.18, three other built-in chains are also supported: INPUT (for packets coming into the box itself), FORWARD (for altering packets being routed through the box), and POSTROUTING (for altering packets as they are about to go out).<br>mangle主要用于专门的数据包更改，包含 PREROUTING，INPUT，OUTPUT，FORWARD，POSTROUTING 五个chain，注意有内核版本区别，只是现在基本没那么老的内核了<br>raw:<br>This table is used mainly for configuring exemptions from connection tracking in combination with the NOTRACK target. It registers at the netfilter hooks with higher priority and is thus called before ip_conntrack, or any other IP tables. It provides the following built-in chains: PREROUTING (for packets arriving via any network interface) OUTPUT (for packets generated by local processes)<br>包含 PREROUTING和OUTPUT，主要用来配置例外<br>chain内部会有实际的rule规则，这个具体后面可以介绍</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>折腾</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>折腾</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>来看下google最新力作Antigravity的水平如何</title>
|
|
<url>/2025/11/23/%E6%9D%A5%E7%9C%8B%E4%B8%8Bgoogle%E6%9C%80%E6%96%B0%E5%8A%9B%E4%BD%9Cgemini-3-pro%E7%9A%84%E6%B0%B4%E5%B9%B3%E5%A6%82%E4%BD%95/</url>
|
|
<content><![CDATA[<p>这是一个类似于cursor的智能ide，并且内置了最新的gemini-3-pro<br><img data-src="https://img.nicksxs.me/uPic/qI56HV.png"><br>界面也是类似于cursor，但是需要一些网络技巧，这个自行解决哈<br>我们还是用一样的prompt的，</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">帮我生成一个todo应用，基于react实现，需要具有非常精美的UI，媲美Instagram那样的网站</span><br></pre></td></tr></table></figure>
|
|
<p>首先也是会生成plan，接着就执行，<br><img data-src="https://img.nicksxs.me/uPic/bP08uw.png"><br>生成的还是比较顺利的，也没有修改，但是它的有个优势点是会自行开启一个chrome，对页面进行dom检查和截图检查<br>并且还会生成一个Walkthrough，包括截图和验证结果<br><img data-src="https://img.nicksxs.me/uPic/Tstgqh.png"><br>只是这个生成的感觉过分的为了保持UI的相似性，或者说可能从语义理解来说感觉有点问题，首先得是个todo应用，只是样式像Instagram<br>因为也有因为中断导致最终是两次生成，我又重新生成了一次，但貌似语义理解的确是个问题<br><img data-src="https://img.nicksxs.me/uPic/V9X7Jv.png"><br>当然生成的效果跟Instagram的相似度是最高的，可能还需要再学习下</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>新奇体验-记我的博客图片被恶意盗刷</title>
|
|
<url>/2025/03/23/%E6%96%B0%E5%A5%87%E4%BD%93%E9%AA%8C-%E8%AE%B0%E6%88%91%E7%9A%84%E5%8D%9A%E5%AE%A2%E5%9B%BE%E7%89%87%E8%A2%AB%E6%81%B6%E6%84%8F%E7%9B%97%E5%88%B7/</url>
|
|
<content><![CDATA[<p>上周四，突然接到腾讯云的电话，当然不是叫我去面试的，是说我账户欠费了，而且一下欠了二十多，因为这个账户我除了包年包月的我都会及时续费，剩下的就是一个放我博客图片的cos对象存储，本来可能一天就几分钱，一年充个十块钱都足够用了，突然一天就欠了二十多还是引起了我的警觉，去看了下说是什么外网下行流量费，我想着怎么又有这么个收费名目了，腾讯云给我的感觉就是一直巧立名目，或者增加收费项，后面也问了客服，说是我开的是公有读私有写，而且允许空refer访问（这个我记得是改过的，不知道为啥现在变成了空refer），也挺奇怪的，这种page托管的博客，不都是只能公有读么，私有读的话我还放啥图片，除非全都走服务端中转。<br>唯一一个优点是总算有个日志，结果看了下是江苏泰顺一哥们一直在换着ip刷，用的还是固定的okhttp客户端，这里想放个图，但是这个优点也被后面的缺点给淹没了，因为腾讯云不支持按访问客户端和ip段设置黑名单，简直离谱，都立了这个名目收费了，对应的功能还不行，都能怀疑是故意漏个口子让刷<br>可能暂时就不能放出来图片了，主要是也没精力搞什么服务端加密，打算迁到r2或者其他方式了，如果有大佬对这方面有经验又碰巧看到了这篇文章，希望能给下指导。<br>还有这位用了江苏泰顺ip（说不定是肉鸡）的哥们，是有啥想不开的呢</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>体验</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>恶意盗刷</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>来看下我装备了5060TI显卡的gpt-oss模型表现</title>
|
|
<url>/2025/11/30/%E6%9D%A5%E7%9C%8B%E4%B8%8B%E6%88%91%E8%A3%85%E5%A4%87%E4%BA%865060TI%E6%98%BE%E5%8D%A1%E7%9A%84gpt-oss%E6%A8%A1%E5%9E%8B%E8%A1%A8%E7%8E%B0/</url>
|
|
<content><![CDATA[<p>之前在我的3060笔记本上试了有显卡的情况下gpt-oss的表现，只能说勉强可以用，比mbp上是可用了很多，毕竟那玩意除非完全把内存都让给gpt-oss，不然都跑不起来，只是生成速度还是有点感人，差不多就4.66token/s,一直觉得能在本地跑个稍微能用点的模型还是种比较不错的体验，所以在最近买了个5060TI，因为这个是最便宜的16G显存的家用显卡了，当然排除各种魔改卡，比如镭7这种，当然喜欢折腾的也可以买来玩玩，硬件上要玩起来还是需要比较多时间的<br>今天用同样的prompt来再对比测试下<br>3060 6g笔记本显卡<br>prompt是<br><code>帮我用react写一个todo应用，样式要美观精致</code><br>我们先对比下效果<br><img data-src="https://img.nicksxs.me/uPic/sCyypJ.png"><br>生成的还是比较简介，重要的是正确的</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">1.77 token/s</span><br><span class="line"></span><br><span class="line">•</span><br><span class="line"></span><br><span class="line">2926 token</span><br><span class="line"></span><br><span class="line">•</span><br><span class="line"></span><br><span class="line">首个token用时 2.21 s</span><br><span class="line"></span><br><span class="line">•</span><br><span class="line"></span><br><span class="line">停止原因: 检测到 EOS token</span><br></pre></td></tr></table></figure>
|
|
<p>对比的是<br>5060ti 16g显卡<br>我们也来看下效果<br><img data-src="https://img.nicksxs.me/uPic/QchgJQ.png"><br>样式稍稍有点问题，但是能一次运行成功</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">27.91 tok/sec</span><br><span class="line"></span><br><span class="line">•</span><br><span class="line"></span><br><span class="line">2236 tokens</span><br><span class="line"></span><br><span class="line">•</span><br><span class="line"></span><br><span class="line">0.41s to first token</span><br><span class="line"></span><br><span class="line">•</span><br><span class="line"></span><br><span class="line">Stop reason: EOS Token Found</span><br></pre></td></tr></table></figure>
|
|
<p>对比两次运行，其实3060在一开始think的时候也耗时比较久，后续生成的速度差异也是比较大的<br>这简单对比比较能看出来能把整个模型权重加载进显存还是有比较大优势的，不过既然本身可以不全加载进显存，我是不是也可以试试32B的模型，毕竟有16+32的显存组合，16显存+32是内存共享的，下次可以体验试试看，比如32B的qwen模型</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>深度学习入门初认识</title>
|
|
<url>/2023/04/30/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E5%85%A5%E9%97%A8%E5%88%9D%E8%AE%A4%E8%AF%86/</url>
|
|
<content><![CDATA[<p>对于深度学习只能说我是个门外汉，开始学习，不过很多还搞不懂，做个记录和分享，基于《深度学习入门：基于 Python 的理论与实现》，<br>第一章 Python 入门就不介绍了，不是重点，不过完全没有 Python 基础的可以看下，我之前算是学过一点点，<br>第二章我觉得入门的方式比较不错，从感知机入手，有一些顾名思义，感知一些状态（输入），来做出反应，简单的就是比如通电了，我的灯就亮起来了，并且往后就是延伸到了计算机的最基础组成，与或门，与门与或门是最基础的，配合非门进行组成，可以作为计算机基础单元ALU 的组成基础，而更复杂的也可以由此进行搭建，这是个比较通俗的解释，根据书中的定义，感知机是具有输入和输出的算法。给定一个输入后，将输出一个既定的值，单层的感知机无法实现更复杂的异或门，但是可以通过 2 层感知机来实现，也就是一个与非门，一个或门，作为第一层的感知机，他们的输出作为与门的输入，就可以成为一个异或门。理论上多层感知机可以表示计算机。<br>而后第三章引出了激活函数，也就是在前面的与或门和基础的感知机的基础上加上了输出的条件，前面与门或门都是最基础的 0,1 游戏，现在可以加上更复杂的判断条件，在输入的基础上配以权重，再加上偏置参数，表示被激活的容易程度，这种激活函数可以被称为阶跃函数，如果超过了一定的值就代表被激活，没有则不激活，但是实际在神经元中被使用的主要是用 sigmoid 函数，相比阶跃函数，sigmoid 函数是一个平滑的曲线，随着输入变化而连续变化，因为相对感知机，神经元需要的信号是连续的实数值信号，再往后则是对输出层的介绍，如果是回归问题，也就是根据输入预测一个（连续的）数值的问题，属于回归问题，可以用恒等函数，而对于分类问题，则使用 softmax 函数，这个函数的一个重要的点在于也是区分于简单的二元分类器，softmax 是将多个结果概率进行数值处理（归一化），也叫做归一化指数函数，对于不同的结果概率是将概率最大的进行放大，凸显其中最大的值并抑制远低于最大值的其他分量。使得其他概率值也能够被使用，但是减弱其份额权重。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>深度学习</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>深度学习</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>用 ollama 本地运行谷歌开源大模型 Gemma</title>
|
|
<url>/2024/03/24/%E7%94%A8-ollama-%E6%9C%AC%E5%9C%B0%E8%BF%90%E8%A1%8C%E5%A4%A7%E6%A8%A1%E5%9E%8B/</url>
|
|
<content><![CDATA[<p>原先在 23 年初的时候调研过一些国产的大模型，包括复旦开源的 MOSS 和清华的 ChatGLM，那时候还是早期版本，需要在 Linux 上，并且有比较好的显卡，而且一般来讲都得是 N 卡，过程中需要安装 pytorch和比较多依赖，并且当时的效果也还比较差，所以后面就没有长期使用。<br>最近看到谷歌在 2 月份开源了大模型 Gemma ，<a href="https://blog.google/technology/developers/gemma-open-models/">gemma</a> 的博客在这里，想要在本地运行这个模型在现在这个阶段也变得简单很多，因为我们有了 ollama 工具<br><img data-src="https://img.nicksxs.me/blog/GmwqJC.png"><br>可以通过这个工具来运行大模型，并且已经支持了谷歌开源的 Gemma<br><img data-src="https://img.nicksxs.me/blog/SRZ4u0.png"><br>我这边本地是 MacBook Pro 14 寸的，m3 pro 的处理器，18g 内存，刚好可以用 7b 量化的模型<br><img data-src="https://img.nicksxs.me/blog/t4GBoS.png"><br>这里有推荐的模型和内存推荐匹配规则，16g 可以运行 13B 及以下模型<br>下载安装完后我们可以用以下命令</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">ollama run gemma:7b</span><br></pre></td></tr></table></figure>
|
|
<p>这里需要拉取模型，约5.2g 大小，考虑网络原因可能会比较慢<br><img data-src="https://img.nicksxs.me/blog/TFh3sQ.png"><br>我们可以简单来试试问个问题<br><img data-src="https://img.nicksxs.me/blog/sbA71p.png"><br><img data-src="https://img.nicksxs.me/blog/0urNCA.png"><br>看出来回答的还是比较丰富的，谷歌出品还是比较有水平的，不至于像 ChatGLM 最初版本的在不做调优的情况下甚至有点前言不搭后语<br>对于想使用 chatgpt 但是没条件，这也算是个低配平替了, 并且已经是个比较可用的了，同时也方便进行学习调优等<br>如果想要类似于 chatgpt 那样的网页版，可以安装 <a href="https://github.com/open-webui/open-webui">open-webui</a><br>可以通过 webui 访问 ollama 运行的大模型，<br>用 docker 启动的命令也贴一下</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main</span><br></pre></td></tr></table></figure>
|
|
<p>不过有个小问题就是 docker 镜像拉取会有点慢，可以添加下国内镜像加速</p>
|
|
<figure class="highlight json"><table><tr><td class="code"><pre><span class="line"><span class="punctuation">{</span></span><br><span class="line"> <span class="attr">"registry-mirrors"</span><span class="punctuation">:</span> <span class="punctuation">[</span></span><br><span class="line"> <span class="string">"https://dockerproxy.com"</span><span class="punctuation">,</span></span><br><span class="line"> <span class="string">"https://docker.mirrors.ustc.edu.cn"</span><span class="punctuation">,</span></span><br><span class="line"> <span class="string">"https://docker.nju.edu.cn"</span></span><br><span class="line"> <span class="punctuation">]</span></span><br><span class="line"><span class="punctuation">}</span></span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/blog/MSh1LI.png"><br>这里有一个小区别，Gemma 在多轮会话的时候会在前面的答案基础上再完善。</p>
|
|
<p>补充一个在 windows 环境下，cpu 跑模型的也是可行的<br><img data-src="https://img.nicksxs.me/blog/cWsGsV.png"><br>现在是大模型可以深入千家万户了，大家都可以尝试下，如果对日常的工作学习有一些效率上的提升也是好的</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>怎么让claude code用国内的模型</title>
|
|
<url>/2025/09/13/%E6%80%8E%E4%B9%88%E8%AE%A9claude-code%E7%94%A8%E5%9B%BD%E5%86%85%E7%9A%84%E6%A8%A1%E5%9E%8B/</url>
|
|
<content><![CDATA[<p>之前体验过gemini code这个工具，谷歌还算比较友好，有token，有网络就可以用，claude code目前是个很严格，而且这家公司也对国内用户很不友好<br>所以就研究了下怎么把它对接上国内的模型<br>幸好之前deepseek发布v3.1的时候天然给了支持</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_">#</span><span class="language-bash"><span class="comment"># 由deepseek提供的兼容claude code的base url</span></span></span><br><span class="line">export ANTHROPIC_BASE_URL=https://api.deepseek.com/anthropic</span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash"><span class="comment"># deepseek的token</span></span></span><br><span class="line">export ANTHROPIC_AUTH_TOKEN={token}</span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash"><span class="comment"># 对应的deepseek-chat模型，这个是非推理的</span></span></span><br><span class="line">export ANTHROPIC_MODEL=deepseek-chat</span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash"><span class="comment"># 快速小模型（同样可设为 deepseek-chat）</span></span></span><br><span class="line">export ANTHROPIC_SMALL_FAST_MODEL=deepseek-chat</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>如果需要设置乘推理型的<br>可以把模型改成</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">export ANTHROPIC_MODEL=deepseek-reasoner</span><br></pre></td></tr></table></figure>
|
|
<p>当然这个如果要长久有效得放在 <code>.zshrc</code> ,根据所用的shell工具来设置<br>然后安装claude需要node版本在18及以上</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">nvm use v18</span><br><span class="line">npm install -g @anthropic-ai/claude-code</span><br></pre></td></tr></table></figure>
|
|
<p>安装以后到想要开发的项目<br>运行</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">claude</span><br></pre></td></tr></table></figure>
|
|
<p>接下来我们同样以todo应用作为测试应用</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">帮我写一个todo应用，使用react，ui需要尽可能美观</span><br></pre></td></tr></table></figure>
|
|
<p>只是加了个美观的要求</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">⏺ Update Todos</span><br><span class="line"> ⎿ ☐ Set up React project structure and dependencies</span><br><span class="line"> ☐ Create main Todo component with state management</span><br><span class="line"> ☐ Design and implement beautiful UI with CSS/styling</span><br><span class="line"> ☐ Add todo functionality (add, delete, toggle complete)</span><br><span class="line"> ☐ Test the application</span><br><span class="line"></span><br><span class="line">⏺ Let me start by checking the current directory structure and then create the React todo application.</span><br></pre></td></tr></table></figure>
|
|
<p>首先它分了五步<br>然后逐步运行对应的动作，生成文件等<br><img data-src="https://img.nicksxs.me/claude_todo_demo.png"><br>生成的还是比较美观可用的，说明国内模型在简单的编码任务上也具有比较不错的能力了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>看完了扫黑风暴,聊聊感想</title>
|
|
<url>/2021/10/24/%E7%9C%8B%E5%AE%8C%E4%BA%86%E6%89%AB%E9%BB%91%E9%A3%8E%E6%9A%B4-%E8%81%8A%E8%81%8A%E6%84%9F%E6%83%B3/</url>
|
|
<content><![CDATA[<p>一直在想这篇怎么写，看了这部剧其实对我的一些观念是有影响的，应该是在 9 月份就看完了，到现在可能才会稍微平静一点，一开始是没有想看这部剧，因为同期有一部差不多同名的电影，被投诉了对湖南埋尸案家属伤害很大，我以为就是投诉的这部电视剧，后来同事跟我说不是，所以就想着看一下，但是没有马上看，因为一直不喜欢追这种比较纠结的剧，当时看人民的名义，就是后面等不了了直接看了小说，所以差不多是等到更完了才看的。</p>
|
|
<p>尝试保持一个比较冷静的状态来聊聊，在看的时候有一点感想就是如果要剧里的坏人排个名，因为明眼看都是孙兴是个穷凶极恶的坏人，干尽了坏事，而且可能是演员表演地好，让人真的恨的牙痒痒，但是更多地还是停留在那些剧情中的表现和他的表情，其实对应的真实案例有更多的，这里尽量不展开，有兴趣可以自行搜索关键字，所以其实我想排个名的话，孙兴的母亲应该是我心目中是造成这个结果的比较大占比的始作俑者，因为是方方面面的，包括对林汉的栽赃迫害，最后串起来是因为他看到了孙兴又出来了，就是那句老话，撒了一个谎以后就要用无数个谎来圆，贺芸为了孙兴，作了第一个恶以后就用了一系列的丧心病狂的操作来保护孙兴，而且这之后所做的事情一件比一件可怕，并且如果不是督导组各种想方设法地去破解谜题，这个事情还可以一直被通过各种操作瞒下去，而孙兴还可以继续地为虎作伥，当然其他的包括高明远以及后面的王政，当然是为了这个操作也提供的各种方式的帮助，甚至是主导了这些操作，但是这里贺芸还是在这个位子上能够通过权力做出非常关键的动作，包括栽赃林汉，并且搞掉了李成阳。其中还有一点是我对剧情设计的质疑，也是我前面提到过一点，因为里面孙兴好像是很爱他的母亲贺芸，似乎想表达的是孙兴作的恶是因为得不到母爱，并且个人感觉如果是一个比较敬爱自己母亲的儿子，似乎应该有所畏惧，对他的行为也会有所限制，不应该变成这样一个无恶不作的恶霸，这也是我一直以来的观点，很多人作恶太多可能是因为没有信仰，不管是信基督耶稣还是信道教佛教，总归有一些制约，当然不是说就绝对不会作恶，只是偏向于有所畏惧敬畏，除了某绿哈。</p>
|
|
<p>而对于其他的人感觉演技都不错，只是最后有一些虎头蛇尾吧，不知道是不是审核的原因，也不细说了怕被请喝茶，还有提一点就是麦佳的这个事情，她其实是里面很惨的一个人，把高明远当成最亲近的人，而其实真相令人感觉不寒而栗，杀父杀母的仇人，对于麦佳这个演员，一直觉得印象深刻，后来才想起来就是在爱情公寓里演被关谷救了要以身相遇的那个女孩，长相其实蛮令人印象深刻的，但好像也一直不温不火，不过也不能说演技很好吧，只是在这里演的任务真的是很可怜了，剧情设计里也应该是个很重要的串联人物，最终被高明远献给了大佬，这里扯开一点，好像有的观点说贺芸之前也是这样的，只是一种推测了。</p>
|
|
<p>看完这部剧其实有很多想说的，但是也为了不被请喝茶，尽量少说了，只想说珍爱生命，还是自己小心吧</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>影评</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>来介绍个源码阅读理解神器-deepwiki</title>
|
|
<url>/2025/04/27/%E6%9D%A5%E4%BB%8B%E7%BB%8D%E4%B8%AA%E6%BA%90%E7%A0%81%E9%98%85%E8%AF%BB%E7%90%86%E8%A7%A3%E7%A5%9E%E5%99%A8-deepwiki/</url>
|
|
<content><![CDATA[<p>上次简单介绍了openmanus的使用，但是它究竟是怎么个原理还是一知半解的，如果想要能比较深入的理解，最直接粗暴的就是阅读源码了，然而对于很多人包括我来说阅读源码不是件简单的事情，有时候会陷入局部细节，不得要领<br>正好这次我发现了有个理解项目的神器，这次不加双引号是因为这个真的好<br>比如我们就拿openmanus举例，首先python的语法就没那么熟，以及对整体结构的理解<br>那么我们就可以打开 openmanus 的仓库地址<br><code>https://github.com/mannaandpoem/OpenManus</code><br>然后把 github 替换成 deepwiki，<br>页面变成了这样<br><img data-src="https://img.nicksxs.me/blog/B4Axbk.png"><br>左边是结构大纲，中间我们可以看到项目的主体介绍，包括有核心的架构图，agent的继承关系，tool的生态系统，LLM的集成<br><img data-src="https://img.nicksxs.me/blog/7JtiDx.png"><br>这里就展示了核心架构，通过这个方式我们如果想对一个项目有个初始的认识，就变得非常简单，因为很多项目的当前的代码都是非常复杂的，没有足够的时间精力是没办法一下子学习到项目的整体结构，因为除非我们是要真正投入到一个项目的开发贡献中，我们大概率都是从了解整体的概况，再分模块的去学习，而这个正是这个deepwiki做得非常牛的地方，相当于给每个项目都加了一个更详细具体的wiki，更牛的还有我们可以通过对话的形式进行提问题，比如我们想自己开发个工具，让openmanus集成进去进行调用<br><img data-src="https://img.nicksxs.me/blog/moQZno.png"><br>它就给出了一个非常详尽的回答，<br>首先</p>
|
|
<ol>
|
|
<li>创建自定义工具类,创建一个基于 <code>BaseTool</code> 的新工具类</li>
|
|
<li>将工具添加到Manus代理,修改<code>Manus</code>类的<code>available_tools</code>定义，将您的工具添加到默认工具列表</li>
|
|
<li>使用MCP协议集成远程工具（可选）</li>
|
|
<li>工具执行流程<br>一旦您的工具被集成，Manus代理会在执行过程中使用它：</li>
|
|
</ol>
|
|
<ul>
|
|
<li>代理的think()方法会向LLM发送请求，包含所有可用工具的信息</li>
|
|
<li>LLM会决定使用哪个工具（包括您的Excel工具）</li>
|
|
<li>代理的act()方法会执行工具调用</li>
|
|
<li>您的工具的execute()方法会被调用，执行Excel函数并返回结果</li>
|
|
<li>结果会被添加到代理的记忆中，用于后续决策<br>这样子就让一个项目的理解跟上手变得非常简单，甚至比如我们想要参与这个项目的开源贡献，也能借助这个 deepwiki 来让我们能快速上手。<br>如果对这个结果不满意还可以开启deep research，能让大模型通过深度思考来给出更加合理的答案，这个deepwiki是目前为止我觉得大模型对程序员最有效的一个工具了。</li>
|
|
</ul>
|
|
]]></content>
|
|
</entry>
|
|
<entry>
|
|
<title>用netty实现一个简单的http server-深入理解下</title>
|
|
<url>/2024/09/15/%E7%94%A8netty%E5%AE%9E%E7%8E%B0%E4%B8%80%E4%B8%AA%E7%AE%80%E5%8D%95%E7%9A%84http-server-%E6%B7%B1%E5%85%A5%E7%90%86%E8%A7%A3%E4%B8%8B/</url>
|
|
<content><![CDATA[<p>上次用netty写的一个玩具http server，发现了一个问题，为啥channelRead0方法会被调用两次，这里我们来研究下<br>我们在收到http请求的时候需要经过的一个必要的过程就是编解码，而这里我们用的是 <code>io.netty.handler.codec.http.HttpServerCodec</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">pipeline.addLast(<span class="keyword">new</span> <span class="title class_">HttpServerCodec</span>());</span><br></pre></td></tr></table></figure>
|
|
<p>它继承了 <code>io.netty.channel.CombinedChannelDuplexHandler</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">HttpServerCodec</span><span class="params">(HttpDecoderConfig config)</span> {</span><br><span class="line"> <span class="built_in">this</span>.queue = <span class="keyword">new</span> <span class="title class_">ArrayDeque</span>();</span><br><span class="line"> <span class="built_in">this</span>.init(<span class="keyword">new</span> <span class="title class_">HttpServerRequestDecoder</span>(config), <span class="keyword">new</span> <span class="title class_">HttpServerResponseEncoder</span>());</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>在初始化的时候调用了 <code>io.netty.channel.CombinedChannelDuplexHandler#init</code><br>传入参数</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">HttpServerCodec</span><span class="params">(HttpDecoderConfig config)</span> {</span><br><span class="line"> <span class="built_in">this</span>.queue = <span class="keyword">new</span> <span class="title class_">ArrayDeque</span>();</span><br><span class="line"> <span class="built_in">this</span>.init(<span class="keyword">new</span> <span class="title class_">HttpServerRequestDecoder</span>(config), <span class="keyword">new</span> <span class="title class_">HttpServerResponseEncoder</span>());</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这个 HttpServerRequestDecoder</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">final</span> <span class="keyword">class</span> <span class="title class_">HttpServerRequestDecoder</span> <span class="keyword">extends</span> <span class="title class_">HttpRequestDecoder</span> {</span><br><span class="line"> HttpServerRequestDecoder(HttpDecoderConfig config) {</span><br><span class="line"> <span class="built_in">super</span>(config);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">decode</span><span class="params">(ChannelHandlerContext ctx, ByteBuf buffer, List<Object> out)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="type">int</span> <span class="variable">oldSize</span> <span class="operator">=</span> out.size();</span><br><span class="line"> <span class="built_in">super</span>.decode(ctx, buffer, out);</span><br><span class="line"> <span class="type">int</span> <span class="variable">size</span> <span class="operator">=</span> out.size();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span>(<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> oldSize; i < size; ++i) {</span><br><span class="line"> <span class="type">Object</span> <span class="variable">obj</span> <span class="operator">=</span> out.get(i);</span><br><span class="line"> <span class="keyword">if</span> (obj <span class="keyword">instanceof</span> HttpRequest) {</span><br><span class="line"> HttpServerCodec.<span class="built_in">this</span>.queue.add(((HttpRequest)obj).method());</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这个decode方法就会调用到父类的decode方法 <code>io.netty.handler.codec.http.HttpObjectDecoder#decode</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">decode</span><span class="params">(ChannelHandlerContext ctx, ByteBuf buffer, List<Object> out)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.resetRequested.get()) {</span><br><span class="line"> <span class="built_in">this</span>.resetNow();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">int</span> toRead;</span><br><span class="line"> <span class="type">int</span> toRead;</span><br><span class="line"> ByteBuf line;</span><br><span class="line"> <span class="keyword">switch</span> (<span class="built_in">this</span>.currentState) {</span><br><span class="line"> <span class="keyword">case</span> SKIP_CONTROL_CHARS:</span><br><span class="line"> <span class="keyword">case</span> READ_INITIAL:</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> line = <span class="built_in">this</span>.lineParser.parse(buffer);</span><br><span class="line"> <span class="keyword">if</span> (line == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> String[] initialLine = <span class="built_in">this</span>.splitInitialLine(line);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">assert</span> initialLine.length == <span class="number">3</span> : <span class="string">"initialLine::length must be 3"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.message = <span class="built_in">this</span>.createMessage(initialLine);</span><br><span class="line"> <span class="built_in">this</span>.currentState = HttpObjectDecoder.State.READ_HEADER;</span><br><span class="line"> } <span class="keyword">catch</span> (Exception var9) {</span><br><span class="line"> out.add(<span class="built_in">this</span>.invalidMessage(<span class="built_in">this</span>.message, buffer, var9));</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">case</span> READ_HEADER:</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">State</span> <span class="variable">nextState</span> <span class="operator">=</span> <span class="built_in">this</span>.readHeaders(buffer);</span><br><span class="line"> <span class="keyword">if</span> (nextState == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.currentState = nextState;</span><br><span class="line"> <span class="keyword">switch</span> (nextState) {</span><br><span class="line"> <span class="keyword">case</span> SKIP_CONTROL_CHARS:</span><br><span class="line"> <span class="built_in">this</span>.addCurrentMessage(out);</span><br><span class="line"> out.add(LastHttpContent.EMPTY_LAST_CONTENT);</span><br><span class="line"> <span class="built_in">this</span>.resetNow();</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> <span class="keyword">case</span> READ_CHUNK_SIZE:</span><br><span class="line"> <span class="keyword">if</span> (!<span class="built_in">this</span>.chunkedSupported) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalArgumentException</span>(<span class="string">"Chunked messages not supported"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.addCurrentMessage(out);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> <span class="keyword">default</span>:</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.contentLength == <span class="number">0L</span> || <span class="built_in">this</span>.contentLength == -<span class="number">1L</span> && <span class="built_in">this</span>.isDecodingRequest()) {</span><br><span class="line"> <span class="built_in">this</span>.addCurrentMessage(out);</span><br><span class="line"> out.add(LastHttpContent.EMPTY_LAST_CONTENT);</span><br><span class="line"> <span class="built_in">this</span>.resetNow();</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">assert</span> nextState == HttpObjectDecoder.State.READ_FIXED_LENGTH_CONTENT || nextState == HttpObjectDecoder.State.READ_VARIABLE_LENGTH_CONTENT;</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.addCurrentMessage(out);</span><br><span class="line"> <span class="keyword">if</span> (nextState == HttpObjectDecoder.State.READ_FIXED_LENGTH_CONTENT) {</span><br><span class="line"> <span class="built_in">this</span>.chunkSize = <span class="built_in">this</span>.contentLength;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Exception var10) {</span><br><span class="line"> out.add(<span class="built_in">this</span>.invalidMessage(<span class="built_in">this</span>.message, buffer, var10));</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">case</span> READ_CHUNK_SIZE:</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> line = <span class="built_in">this</span>.lineParser.parse(buffer);</span><br><span class="line"> <span class="keyword">if</span> (line == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> toRead = getChunkSize(line.array(), line.arrayOffset() + line.readerIndex(), line.readableBytes());</span><br><span class="line"> <span class="built_in">this</span>.chunkSize = (<span class="type">long</span>)toRead;</span><br><span class="line"> <span class="keyword">if</span> (toRead == <span class="number">0</span>) {</span><br><span class="line"> <span class="built_in">this</span>.currentState = HttpObjectDecoder.State.READ_CHUNK_FOOTER;</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.currentState = HttpObjectDecoder.State.READ_CHUNKED_CONTENT;</span><br><span class="line"> } <span class="keyword">catch</span> (Exception var8) {</span><br><span class="line"> out.add(<span class="built_in">this</span>.invalidChunk(buffer, var8));</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">case</span> READ_CHUNKED_CONTENT:</span><br><span class="line"> <span class="keyword">assert</span> <span class="built_in">this</span>.chunkSize <= <span class="number">2147483647L</span>;</span><br><span class="line"></span><br><span class="line"> toRead = Math.min((<span class="type">int</span>)<span class="built_in">this</span>.chunkSize, <span class="built_in">this</span>.maxChunkSize);</span><br><span class="line"> <span class="keyword">if</span> (!<span class="built_in">this</span>.allowPartialChunks && buffer.readableBytes() < toRead) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> toRead = Math.min(toRead, buffer.readableBytes());</span><br><span class="line"> <span class="keyword">if</span> (toRead == <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">HttpContent</span> <span class="variable">chunk</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">DefaultHttpContent</span>(buffer.readRetainedSlice(toRead));</span><br><span class="line"> <span class="built_in">this</span>.chunkSize -= (<span class="type">long</span>)toRead;</span><br><span class="line"> out.add(chunk);</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.chunkSize != <span class="number">0L</span>) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.currentState = HttpObjectDecoder.State.READ_CHUNK_DELIMITER;</span><br><span class="line"> <span class="keyword">case</span> READ_CHUNK_DELIMITER:</span><br><span class="line"> toRead = buffer.writerIndex();</span><br><span class="line"> toRead = buffer.readerIndex();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">while</span>(toRead > toRead) {</span><br><span class="line"> <span class="type">byte</span> <span class="variable">next</span> <span class="operator">=</span> buffer.getByte(toRead++);</span><br><span class="line"> <span class="keyword">if</span> (next == <span class="number">10</span>) {</span><br><span class="line"> <span class="built_in">this</span>.currentState = HttpObjectDecoder.State.READ_CHUNK_SIZE;</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> buffer.readerIndex(toRead);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> <span class="keyword">case</span> READ_VARIABLE_LENGTH_CONTENT:</span><br><span class="line"> toRead = Math.min(buffer.readableBytes(), <span class="built_in">this</span>.maxChunkSize);</span><br><span class="line"> <span class="keyword">if</span> (toRead > <span class="number">0</span>) {</span><br><span class="line"> <span class="type">ByteBuf</span> <span class="variable">content</span> <span class="operator">=</span> buffer.readRetainedSlice(toRead);</span><br><span class="line"> out.add(<span class="keyword">new</span> <span class="title class_">DefaultHttpContent</span>(content));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> <span class="keyword">case</span> READ_FIXED_LENGTH_CONTENT:</span><br><span class="line"> toRead = buffer.readableBytes();</span><br><span class="line"> <span class="keyword">if</span> (toRead == <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> toRead = Math.min(toRead, <span class="built_in">this</span>.maxChunkSize);</span><br><span class="line"> <span class="keyword">if</span> ((<span class="type">long</span>)toRead > <span class="built_in">this</span>.chunkSize) {</span><br><span class="line"> toRead = (<span class="type">int</span>)<span class="built_in">this</span>.chunkSize;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">ByteBuf</span> <span class="variable">content</span> <span class="operator">=</span> buffer.readRetainedSlice(toRead);</span><br><span class="line"> <span class="built_in">this</span>.chunkSize -= (<span class="type">long</span>)toRead;</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.chunkSize == <span class="number">0L</span>) {</span><br><span class="line"> out.add(<span class="keyword">new</span> <span class="title class_">DefaultLastHttpContent</span>(content, <span class="built_in">this</span>.trailersFactory));</span><br><span class="line"> <span class="built_in">this</span>.resetNow();</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> out.add(<span class="keyword">new</span> <span class="title class_">DefaultHttpContent</span>(content));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> <span class="keyword">case</span> READ_CHUNK_FOOTER:</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">LastHttpContent</span> <span class="variable">trailer</span> <span class="operator">=</span> <span class="built_in">this</span>.readTrailingHeaders(buffer);</span><br><span class="line"> <span class="keyword">if</span> (trailer == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> out.add(trailer);</span><br><span class="line"> <span class="built_in">this</span>.resetNow();</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> } <span class="keyword">catch</span> (Exception var7) {</span><br><span class="line"> out.add(<span class="built_in">this</span>.invalidChunk(buffer, var7));</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">case</span> BAD_MESSAGE:</span><br><span class="line"> buffer.skipBytes(buffer.readableBytes());</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">case</span> UPGRADED:</span><br><span class="line"> toRead = buffer.readableBytes();</span><br><span class="line"> <span class="keyword">if</span> (toRead > <span class="number">0</span>) {</span><br><span class="line"> out.add(buffer.readBytes(toRead));</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这个方法里会做实际的byte转换成message或者说string的转换<br>而在我们把消息转换完成，<br><img data-src="https://img.nicksxs.me/blog/HdBuGL.png"><br>会在最后加一个 <code>io.netty.handler.codec.http.LastHttpContent#EMPTY_LAST_CONTENT</code><br>这样其实我的message就会变成两条，也就是为啥channelRead0会被调用两次，这样的目的也是让我的业务代码可以分辨是否请求已经读完，并且可以在读完以后有一些其他操作空间<br>可以看到第一次调用channelRead0的msg<br><img data-src="https://img.nicksxs.me/blog/TtP8YU.png"><br>第二次就是<br><img data-src="https://img.nicksxs.me/blog/JiPTGt.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>来个openmanus的浅入门</title>
|
|
<url>/2025/04/20/%E6%9D%A5%E4%B8%AAopenmanus%E7%9A%84%E6%B5%85%E5%85%A5%E9%97%A8/</url>
|
|
<content><![CDATA[<p>前阵子一个manus在目前的所谓人工智能圈子里甚至普通人视野里都很火了，宣称是什么中国的下一个deepseek时刻，首先deepseek是经过了v1，v2等一系列版本的迭代之后，并且一直是在技术上非常花功夫的，有种宝剑锋从磨砺出的感觉，而这个manus听着更像是个蹭热度的<br>这不没出多久有个openmanus宣称用了三小时做了个开源的复刻版，那么我们就来简单体验下，从概念上来说吧，有点类似于做了个mcp的规划和整合调用<br>我个人理解好像没有到改变世界的程度<br>首先呢我们先来安装下<br>可以使用conda，也可以使用uv，以conda举例，先建个环境</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">conda create -n open_manus python=3.12</span><br><span class="line">conda activate open_manus</span><br></pre></td></tr></table></figure>
|
|
<p>然后来clone下代码仓库</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">git clone https://github.com/mannaandpoem/OpenManus.git</span><br><span class="line">cd OpenManus</span><br></pre></td></tr></table></figure>
|
|
<p>接着再安装下依赖</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">pip install -r requirements.txt</span><br></pre></td></tr></table></figure>
|
|
<p>这里可以借助下源替换加速，临时使用可以这样子 <code>pip install -i https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple some-package </code> , 这里需要安装蛮久的，可能也说明了这是个依赖于很多现成库的工具</p>
|
|
<h3 id="第二阶段配置"><a href="#第二阶段配置" class="headerlink" title="第二阶段配置"></a>第二阶段配置</h3><p>配置其实也是常规的，依赖于大模型，那么要不就是自己部署提供api，要不就是去火山引擎或者其他大模型api提供商搞个api（免费额度用完要自己付费的）</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">cp config/config.example.toml config/config.toml</span><br></pre></td></tr></table></figure>
|
|
<p>这一步其实跟之前使用chatbox连接火山的配置类似<br>因为像国外很多都还是openai的接口服务，这边就需要改用成国内可用的</p>
|
|
<figure class="highlight toml"><table><tr><td class="code"><pre><span class="line"><span class="section">[llm]</span></span><br><span class="line"><span class="attr">model</span> = <span class="string">"deepseek-r1-250120"</span> <span class="comment"># The LLM model to use</span></span><br><span class="line"><span class="attr">base_url</span> = <span class="string">"https://ark.cn-beijing.volces.com/api/v3"</span> <span class="comment"># API endpoint URL</span></span><br><span class="line"><span class="attr">api_key</span> = <span class="string">"xxxxxxxxxxxxxxxxxx"</span> <span class="comment"># Your API key</span></span><br><span class="line"><span class="attr">max_tokens</span> = <span class="number">8192</span> <span class="comment"># Maximum number of tokens in the response</span></span><br><span class="line"><span class="attr">temperature</span> = <span class="number">0.0</span> </span><br></pre></td></tr></table></figure>
|
|
<p>模型可以用deepseek-r1，不过要注意是什么时间版本的，否则也会访问不到<br>然后我们就可以运行</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">python main.py</span><br></pre></td></tr></table></figure>
|
|
<p>来运行openmanus，我们简单问个问题</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">明天杭州的天气怎么样，给出个穿衣指南</span><br></pre></td></tr></table></figure>
|
|
<p>看下结果</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">python main.py</span><br><span class="line">INFO [browser_use] BrowserUse logging setup complete with level info</span><br><span class="line">INFO [root] Anonymized telemetry enabled. See https://docs.browser-use.com/development/telemetry for more information.</span><br><span class="line">Enter your prompt: 帮我看下明天杭州的天气，并且生成穿衣指南</span><br><span class="line">2025-04-20 20:40:14.721 | WARNING | __main__:main:16 - Processing your request...</span><br><span class="line">2025-04-20 20:40:14.722 | INFO | app.agent.base:run:140 - Executing step 1/20</span><br><span class="line">2025-04-20 20:40:33.488 | INFO | app.llm:update_token_count:250 - Token usage: Input=2165, Completion=658, Cumulative Input=2165, Cumulative Completion=658, Total=2823, Cumulative Total=2823</span><br><span class="line">2025-04-20 20:40:33.489 | INFO | app.agent.toolcall:think:81 - ✨ Manus's thoughts:</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">2025-04-20 20:40:33.489 | INFO | app.agent.toolcall:think:82 - 🛠️ Manus selected 1 tools to use</span><br><span class="line">2025-04-20 20:40:33.489 | INFO | app.agent.toolcall:think:86 - 🧰 Tools being prepared: ['browser_use']</span><br><span class="line">2025-04-20 20:40:33.489 | INFO | app.agent.toolcall:think:89 - 🔧 Tool arguments: {</span><br><span class="line"> "action": "web_search",</span><br><span class="line"> "query": "杭州明天天气预报"</span><br><span class="line">}</span><br><span class="line">2025-04-20 20:40:33.490 | INFO | app.agent.toolcall:execute_tool:180 - 🔧 Activating tool: 'browser_use'...</span><br><span class="line">ERROR [browser] Failed to initialize Playwright browser: BrowserType.launch: Executable doesn't exist at /Users/username/Library/Caches/ms-playwright/chromium-1161/chrome-mac/Chromium.app/Contents/MacOS/Chromium</span><br><span class="line">╔════════════════════════════════════════════════════════════╗</span><br><span class="line">║ Looks like Playwright was just installed or updated. ║</span><br><span class="line">║ Please run the following command to download new browsers: ║</span><br><span class="line">║ ║</span><br><span class="line">║ playwright install ║</span><br><span class="line">║ ║</span><br><span class="line">║ <3 Playwright Team ║</span><br><span class="line">╚════════════════════════════════════════════════════════════╝</span><br><span class="line">2025-04-20 20:40:34.896 | INFO | app.agent.toolcall:act:150 - 🎯 Tool 'browser_use' completed its mission! Result: Observed output of cmd `browser_use` executed:</span><br><span class="line">Error: Browser action 'web_search' failed: BrowserType.launch: Executable doesn't exist at /Users/username/Library/Caches/ms-playwright/chromium-1161/chrome-mac/Chromium.app/Contents/MacOS/Chromium</span><br><span class="line">╔════════════════════════════════════════════════════════════╗</span><br><span class="line">║ Looks like Playwright was just installed or updated. ║</span><br><span class="line">║ Please run the following command to download new browsers: ║</span><br><span class="line">║ ║</span><br><span class="line">║ playwright install ║</span><br><span class="line">║ ║</span><br><span class="line">║ <3 Playwright Team ║</span><br><span class="line">╚════════════════════════════════════════════════════════════╝</span><br><span class="line">2025-04-20 20:40:34.896 | INFO | app.agent.base:run:140 - Executing step 2/20</span><br><span class="line">ERROR [browser] Failed to initialize Playwright browser: BrowserType.launch: Executable doesn't exist at /Users/username/Library/Caches/ms-playwright/chromium-1161/chrome-mac/Chromium.app/Contents/MacOS/Chromium</span><br><span class="line">╔════════════════════════════════════════════════════════════╗</span><br><span class="line">║ Looks like Playwright was just installed or updated. ║</span><br><span class="line">║ Please run the following command to download new browsers: ║</span><br><span class="line">║ ║</span><br><span class="line">║ playwright install ║</span><br><span class="line">║ ║</span><br><span class="line">║ <3 Playwright Team ║</span><br><span class="line">╚════════════════════════════════════════════════════════════╝</span><br><span class="line">WARNING [browser] Page load failed, continuing...</span><br><span class="line">ERROR [browser] Failed to initialize Playwright browser: BrowserType.launch: Executable doesn't exist at /Users/username/Library/Caches/ms-playwright/chromium-1161/chrome-mac/Chromium.app/Contents/MacOS/Chromium</span><br><span class="line">╔════════════════════════════════════════════════════════════╗</span><br><span class="line">║ Looks like Playwright was just installed or updated. ║</span><br><span class="line">║ Please run the following command to download new browsers: ║</span><br><span class="line">║ ║</span><br><span class="line">║ playwright install ║</span><br><span class="line">║ ║</span><br><span class="line">║ <3 Playwright Team ║</span><br><span class="line">╚════════════════════════════════════════════════════════════╝</span><br></pre></td></tr></table></figure>
|
|
<p>根据这个返回可以看到它做了些啥，主要是规划步骤和选择调用的工具，相对来说没有特别的，playwright我就不去搞了，浏览器那套</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>用netty实现一个简单的http server</title>
|
|
<url>/2024/09/08/%E7%94%A8netty%E5%AE%9E%E7%8E%B0%E4%B8%80%E4%B8%AA%E7%AE%80%E5%8D%95%E7%9A%84http-server/</url>
|
|
<content><![CDATA[<p>netty是java网络框架中非常有名，因为它把各种网络类型，阻塞非阻塞的都做了上层的抽象，非常值得学习，这边就先以一个简单的http server来做下实践，<br>主体的server代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> <span class="keyword">throws</span> InterruptedException {</span><br><span class="line"></span><br><span class="line"> <span class="comment">// boss 线程组</span></span><br><span class="line"> <span class="type">EventLoopGroup</span> <span class="variable">boss</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">NioEventLoopGroup</span>();</span><br><span class="line"> <span class="comment">// worker 线程组</span></span><br><span class="line"> <span class="type">EventLoopGroup</span> <span class="variable">work</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">NioEventLoopGroup</span>();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 服务端启动类</span></span><br><span class="line"> <span class="type">ServerBootstrap</span> <span class="variable">bootstrap</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ServerBootstrap</span>();</span><br><span class="line"> <span class="comment">// 绑定线程组</span></span><br><span class="line"> bootstrap.group(boss, work)</span><br><span class="line"> <span class="comment">// 设置服务端的tcp连接的最大排队连接数</span></span><br><span class="line"> .option(ChannelOption.SO_BACKLOG, <span class="number">1024</span>)</span><br><span class="line"> <span class="comment">// 设置日志处理器</span></span><br><span class="line"> .handler(<span class="keyword">new</span> <span class="title class_">LoggingHandler</span>(LogLevel.DEBUG))</span><br><span class="line"> <span class="comment">// 绑定Channel</span></span><br><span class="line"> .channel(NioServerSocketChannel.class)</span><br><span class="line"> <span class="comment">// 实际的逻辑handler</span></span><br><span class="line"> .childHandler(<span class="keyword">new</span> <span class="title class_">HttpServerInitializer</span>());</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 绑定端口</span></span><br><span class="line"> <span class="type">ChannelFuture</span> <span class="variable">f</span> <span class="operator">=</span> bootstrap.bind(<span class="keyword">new</span> <span class="title class_">InetSocketAddress</span>(<span class="number">8082</span>)).sync();</span><br><span class="line"> System.out.println(<span class="string">"server start up on port 8080"</span>);</span><br><span class="line"> f.channel().closeFuture().sync();</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> boss.shutdownGracefully();</span><br><span class="line"> work.shutdownGracefully();</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后来看下 HttpServerInitializer</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">HttpServerInitializer</span> <span class="keyword">extends</span> <span class="title class_">ChannelInitializer</span><SocketChannel> {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">initChannel</span><span class="params">(SocketChannel socketChannel)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="type">ChannelPipeline</span> <span class="variable">pipeline</span> <span class="operator">=</span> socketChannel.pipeline();</span><br><span class="line"> <span class="comment">// 在这个Channel的pipeline上添加处理器</span></span><br><span class="line"> <span class="comment">// 前三个都是netty提供的</span></span><br><span class="line"> pipeline.addLast(<span class="keyword">new</span> <span class="title class_">HttpServerCodec</span>());</span><br><span class="line"> pipeline.addLast(<span class="keyword">new</span> <span class="title class_">HttpContentCompressor</span>((CompressionOptions[]) <span class="literal">null</span> ));</span><br><span class="line"> pipeline.addLast(<span class="keyword">new</span> <span class="title class_">HttpServerExpectContinueHandler</span>());</span><br><span class="line"> <span class="comment">// 这个是我们自己实现的逻辑</span></span><br><span class="line"> pipeline.addLast(<span class="keyword">new</span> <span class="title class_">SimpleHttpServerHandler</span>());</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>实际的代码也是很简略，我们用一个hello world统一响应所有的请求</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">SimpleHttpServerHandler</span> <span class="keyword">extends</span> <span class="title class_">SimpleChannelInboundHandler</span><HttpObject> {</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">byte</span>[] CONTENT = { <span class="string">'H'</span>, <span class="string">'e'</span>, <span class="string">'l'</span>, <span class="string">'l'</span>, <span class="string">'o'</span>, <span class="string">' '</span>, <span class="string">'W'</span>, <span class="string">'o'</span>, <span class="string">'r'</span>, <span class="string">'l'</span>, <span class="string">'d'</span> };</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">channelReadComplete</span><span class="params">(ChannelHandlerContext ctx)</span> {</span><br><span class="line"> <span class="comment">// 刷写响应</span></span><br><span class="line"> ctx.flush();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">channelRead0</span><span class="params">(ChannelHandlerContext channelHandlerContext, HttpObject msg)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="comment">// 先是从这个Channel读取到内容</span></span><br><span class="line"> System.out.println(<span class="string">"channel read"</span>);</span><br><span class="line"> <span class="keyword">if</span> (msg <span class="keyword">instanceof</span> HttpRequest) {</span><br><span class="line"> <span class="comment">// 判断是否为HttpRequest，这个其实依赖于前面netty自带的编码器</span></span><br><span class="line"> <span class="type">HttpRequest</span> <span class="variable">httpRequest</span> <span class="operator">=</span> (HttpRequest) msg;</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">keepAlive</span> <span class="operator">=</span> HttpUtil.isKeepAlive(httpRequest);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 包装http响应</span></span><br><span class="line"> <span class="type">FullHttpResponse</span> <span class="variable">response</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">DefaultFullHttpResponse</span>(httpRequest.protocolVersion(), OK, Unpooled.wrappedBuffer(CONTENT));</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 设置请求头</span></span><br><span class="line"> response.headers()</span><br><span class="line"> .set(CONTENT_TYPE, TEXT_PLAIN)</span><br><span class="line"> .setInt(CONTENT_LENGTH, response.content().readableBytes());</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 设置是否要连接保活</span></span><br><span class="line"> <span class="keyword">if</span> (keepAlive) {</span><br><span class="line"> <span class="keyword">if</span> (!httpRequest.protocolVersion().isKeepAliveDefault()) {</span><br><span class="line"> response.headers().set(CONNECTION, KEEP_ALIVE);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> response.headers().set(CONNECTION, CLOSE);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 写入channelFuture</span></span><br><span class="line"> <span class="type">ChannelFuture</span> <span class="variable">future</span> <span class="operator">=</span> channelHandlerContext.write(response);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 如果不保活，添加监听器</span></span><br><span class="line"> <span class="keyword">if</span> (!keepAlive) {</span><br><span class="line"> future.addListener(ChannelFutureListener.CLOSE);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">exceptionCaught</span><span class="params">(ChannelHandlerContext ctx, Throwable cause)</span> {</span><br><span class="line"> cause.printStackTrace();</span><br><span class="line"> ctx.close();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>上面就是这个简单的示例了，只是我在 <code>channelRead0</code> 入口建了个打印，但是这个再接收请求时会打印两次，可以作为个探讨题，我们下次来分析下</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>用gpt-oss零改动实现一个todo应用</title>
|
|
<url>/2025/09/07/%E7%94%A8gpt-oss%E9%9B%B6%E6%94%B9%E5%8A%A8%E5%AE%9E%E7%8E%B0%E4%B8%80%E4%B8%AAtodo%E5%BA%94%E7%94%A8/</url>
|
|
<content><![CDATA[<h2 id="上次是文字型的，这次我们来写点代码试试，看看效果怎么样首先肯定是比较简单的尝试开始写一个todo应用，主要是看能不能零改动就跑起来用最简单的提示词然后就给出了代码"><a href="#上次是文字型的，这次我们来写点代码试试，看看效果怎么样首先肯定是比较简单的尝试开始写一个todo应用，主要是看能不能零改动就跑起来用最简单的提示词然后就给出了代码" class="headerlink" title="上次是文字型的，这次我们来写点代码试试，看看效果怎么样首先肯定是比较简单的尝试开始写一个todo应用，主要是看能不能零改动就跑起来用最简单的提示词然后就给出了代码"></a>上次是文字型的，这次我们来写点代码试试，看看效果怎么样<br>首先肯定是比较简单的尝试开始<br>写一个todo应用，主要是看能不能零改动就跑起来<br>用最简单的提示词<br><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">帮我写一个todo应用，基于react</span><br></pre></td></tr></table></figure><br>然后就给出了代码</h2><p>下面给你一个 <strong>最小可运行</strong> 的 Todo 应用示例，基于 <strong>React + Vite</strong>（如果你习惯 <code>create-react-app</code>，思路几乎完全一样，只是文件夹结构略有差异）。<br>应用包含：</p>
|
|
<table>
|
|
<thead>
|
|
<tr>
|
|
<th>组件</th>
|
|
<th>作用</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr>
|
|
<td><code>TodoInput</code></td>
|
|
<td>新增条目输入框</td>
|
|
</tr>
|
|
<tr>
|
|
<td><code>TodoItem</code></td>
|
|
<td>单个待办项（支持完成/取消、删除）</td>
|
|
</tr>
|
|
<tr>
|
|
<td><code>TodoList</code></td>
|
|
<td>渲染 Todo 列表</td>
|
|
</tr>
|
|
<tr>
|
|
<td><code>TodoApp</code>/<code>App</code></td>
|
|
<td>主容器，维护全局 <code>todos</code> 状态</td>
|
|
</tr>
|
|
</tbody></table>
|
|
<blockquote>
|
|
<p><strong>安装依赖</strong></p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">npm create vite@latest todo-app --template react</span><br><span class="line">cd todo-app</span><br><span class="line">npm i</span><br><span class="line">npm run dev</span><br></pre></td></tr></table></figure>
|
|
</blockquote>
|
|
<p>下面按文件展示完整代码（把对应文件替换或新建即可）。</p>
|
|
<hr>
|
|
<h2 id="1️⃣-src-main-jsx"><a href="#1️⃣-src-main-jsx" class="headerlink" title="1️⃣ src/main.jsx"></a>1️⃣ <code>src/main.jsx</code></h2><figure class="highlight jsx"><table><tr><td class="code"><pre><span class="line"><span class="keyword">import</span> <span class="title class_">React</span> <span class="keyword">from</span> <span class="string">"react"</span>;</span><br><span class="line"><span class="keyword">import</span> { createRoot } <span class="keyword">from</span> <span class="string">"react-dom/client"</span>;</span><br><span class="line"><span class="keyword">import</span> <span class="title class_">App</span> <span class="keyword">from</span> <span class="string">"./App"</span>;</span><br><span class="line"><span class="keyword">import</span> <span class="string">"./index.css"</span>;</span><br><span class="line"></span><br><span class="line"><span class="title function_">createRoot</span>(<span class="variable language_">document</span>.<span class="title function_">getElementById</span>(<span class="string">"root"</span>)).<span class="title function_">render</span>(<span class="language-xml"><span class="tag"><<span class="name">App</span> /></span></span>);</span><br></pre></td></tr></table></figure>
|
|
|
|
<hr>
|
|
<h2 id="2️⃣-src-App-jsx"><a href="#2️⃣-src-App-jsx" class="headerlink" title="2️⃣ src/App.jsx"></a>2️⃣ <code>src/App.jsx</code></h2><figure class="highlight jsx"><table><tr><td class="code"><pre><span class="line"><span class="keyword">import</span> <span class="title class_">React</span>, { useState, useEffect } <span class="keyword">from</span> <span class="string">"react"</span>;</span><br><span class="line"><span class="keyword">import</span> <span class="title class_">TodoInput</span> <span class="keyword">from</span> <span class="string">"./components/TodoInput"</span>;</span><br><span class="line"><span class="keyword">import</span> <span class="title class_">TodoList</span> <span class="keyword">from</span> <span class="string">"./components/TodoList"</span>;</span><br><span class="line"><span class="keyword">import</span> <span class="string">"./App.css"</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 关键点：</span></span><br><span class="line"><span class="comment"> * 1. 用 useState 存放 todos</span></span><br><span class="line"><span class="comment"> * 2. 用 useEffect 做本地存储同步，仅在组件挂载时读取一次</span></span><br><span class="line"><span class="comment"> * 3. 处理增删改的通用方法</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">function</span> <span class="title function_">App</span>(<span class="params"></span>) {</span><br><span class="line"> <span class="keyword">const</span> [todos, setTodos] = <span class="title function_">useState</span>([]);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 从 localStorage 初始化</span></span><br><span class="line"> <span class="title function_">useEffect</span>(<span class="function">() =></span> {</span><br><span class="line"> <span class="keyword">const</span> stored = <span class="variable language_">localStorage</span>.<span class="title function_">getItem</span>(<span class="string">"todos"</span>);</span><br><span class="line"> <span class="keyword">if</span> (stored) {</span><br><span class="line"> <span class="title function_">setTodos</span>(<span class="title class_">JSON</span>.<span class="title function_">parse</span>(stored));</span><br><span class="line"> }</span><br><span class="line"> }, []);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 每次 todos 变更时同步到 localStorage</span></span><br><span class="line"> <span class="title function_">useEffect</span>(<span class="function">() =></span> {</span><br><span class="line"> <span class="variable language_">localStorage</span>.<span class="title function_">setItem</span>(<span class="string">"todos"</span>, <span class="title class_">JSON</span>.<span class="title function_">stringify</span>(todos));</span><br><span class="line"> }, [todos]);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">const</span> <span class="title function_">addTodo</span> = (<span class="params">title</span>) => {</span><br><span class="line"> <span class="keyword">const</span> newTodo = {</span><br><span class="line"> <span class="attr">id</span>: <span class="title class_">Date</span>.<span class="title function_">now</span>(),</span><br><span class="line"> title,</span><br><span class="line"> <span class="attr">completed</span>: <span class="literal">false</span>,</span><br><span class="line"> };</span><br><span class="line"> <span class="title function_">setTodos</span>([newTodo, ...todos]);</span><br><span class="line"> };</span><br><span class="line"></span><br><span class="line"> <span class="keyword">const</span> <span class="title function_">toggleComplete</span> = (<span class="params">id</span>) => {</span><br><span class="line"> <span class="title function_">setTodos</span>(<span class="function">(<span class="params">prev</span>) =></span></span><br><span class="line"> prev.<span class="title function_">map</span>(<span class="function">(<span class="params">t</span>) =></span></span><br><span class="line"> t.<span class="property">id</span> === id ? { ...t, <span class="attr">completed</span>: !t.<span class="property">completed</span> } : t</span><br><span class="line"> )</span><br><span class="line"> );</span><br><span class="line"> };</span><br><span class="line"></span><br><span class="line"> <span class="keyword">const</span> <span class="title function_">deleteTodo</span> = (<span class="params">id</span>) => {</span><br><span class="line"> <span class="title function_">setTodos</span>(<span class="function">(<span class="params">prev</span>) =></span> prev.<span class="title function_">filter</span>(<span class="function">(<span class="params">t</span>) =></span> t.<span class="property">id</span> !== id));</span><br><span class="line"> };</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> (</span><br><span class="line"> <span class="language-xml"><span class="tag"><<span class="name">div</span> <span class="attr">className</span>=<span class="string">"app-container"</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">h1</span>></span>Todo List<span class="tag"></<span class="name">h1</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">TodoInput</span> <span class="attr">addTodo</span>=<span class="string">{addTodo}</span> /></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">TodoList</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">todos</span>=<span class="string">{todos}</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">toggleComplete</span>=<span class="string">{toggleComplete}</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">deleteTodo</span>=<span class="string">{deleteTodo}</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> /></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">div</span>></span></span></span><br><span class="line"> );</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">export</span> <span class="keyword">default</span> <span class="title class_">App</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<hr>
|
|
<h2 id="3️⃣-src-components-TodoInput-jsx"><a href="#3️⃣-src-components-TodoInput-jsx" class="headerlink" title="3️⃣ src/components/TodoInput.jsx"></a>3️⃣ <code>src/components/TodoInput.jsx</code></h2><figure class="highlight jsx"><table><tr><td class="code"><pre><span class="line"><span class="keyword">import</span> <span class="title class_">React</span>, { useState } <span class="keyword">from</span> <span class="string">"react"</span>;</span><br><span class="line"><span class="keyword">import</span> <span class="string">"./TodoInput.css"</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title function_">TodoInput</span> = (<span class="params">{ addTodo }</span>) => {</span><br><span class="line"> <span class="keyword">const</span> [value, setValue] = <span class="title function_">useState</span>(<span class="string">""</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">const</span> <span class="title function_">handleSubmit</span> = (<span class="params">e</span>) => {</span><br><span class="line"> e.<span class="title function_">preventDefault</span>();</span><br><span class="line"> <span class="keyword">const</span> trimmed = value.<span class="title function_">trim</span>();</span><br><span class="line"> <span class="keyword">if</span> (trimmed) {</span><br><span class="line"> <span class="title function_">addTodo</span>(trimmed);</span><br><span class="line"> <span class="title function_">setValue</span>(<span class="string">""</span>);</span><br><span class="line"> }</span><br><span class="line"> };</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> (</span><br><span class="line"> <span class="language-xml"><span class="tag"><<span class="name">form</span> <span class="attr">className</span>=<span class="string">"todo-input"</span> <span class="attr">onSubmit</span>=<span class="string">{handleSubmit}</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">input</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">type</span>=<span class="string">"text"</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">placeholder</span>=<span class="string">"Add a new todo..."</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">value</span>=<span class="string">{value}</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">onChange</span>=<span class="string">{(e)</span> =></span> setValue(e.target.value)}</span></span><br><span class="line"><span class="language-xml"> /></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">button</span> <span class="attr">type</span>=<span class="string">"submit"</span>></span>Add<span class="tag"></<span class="name">button</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">form</span>></span></span></span><br><span class="line"> );</span><br><span class="line">};</span><br><span class="line"></span><br><span class="line"><span class="keyword">export</span> <span class="keyword">default</span> <span class="title class_">TodoInput</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<hr>
|
|
<h2 id="4️⃣-src-components-TodoItem-jsx"><a href="#4️⃣-src-components-TodoItem-jsx" class="headerlink" title="4️⃣ src/components/TodoItem.jsx"></a>4️⃣ <code>src/components/TodoItem.jsx</code></h2><figure class="highlight jsx"><table><tr><td class="code"><pre><span class="line"><span class="keyword">import</span> <span class="title class_">React</span> <span class="keyword">from</span> <span class="string">"react"</span>;</span><br><span class="line"><span class="keyword">import</span> <span class="string">"./TodoItem.css"</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title function_">TodoItem</span> = (<span class="params">{ todo, toggleComplete, deleteTodo }</span>) => {</span><br><span class="line"> <span class="keyword">return</span> (</span><br><span class="line"> <span class="language-xml"><span class="tag"><<span class="name">div</span> <span class="attr">className</span>=<span class="string">"todo-item"</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">label</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">input</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">type</span>=<span class="string">"checkbox"</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">checked</span>=<span class="string">{todo.completed}</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">onChange</span>=<span class="string">{()</span> =></span> toggleComplete(todo.id)}</span></span><br><span class="line"><span class="language-xml"> /></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">span</span> <span class="attr">className</span>=<span class="string">{todo.completed</span> ? "<span class="attr">completed</span>" <span class="attr">:</span> ""}></span></span></span><br><span class="line"><span class="language-xml"> {todo.title}</span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">span</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">label</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">button</span> <span class="attr">className</span>=<span class="string">"delete-btn"</span> <span class="attr">onClick</span>=<span class="string">{()</span> =></span> deleteTodo(todo.id)}></span></span><br><span class="line"><span class="language-xml"> ×</span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">button</span>></span></span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">div</span>></span></span></span><br><span class="line"> );</span><br><span class="line">};</span><br><span class="line"></span><br><span class="line"><span class="keyword">export</span> <span class="keyword">default</span> <span class="title class_">TodoItem</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<hr>
|
|
<h2 id="5️⃣-src-components-TodoList-jsx"><a href="#5️⃣-src-components-TodoList-jsx" class="headerlink" title="5️⃣ src/components/TodoList.jsx"></a>5️⃣ <code>src/components/TodoList.jsx</code></h2><figure class="highlight jsx"><table><tr><td class="code"><pre><span class="line"><span class="keyword">import</span> <span class="title class_">React</span> <span class="keyword">from</span> <span class="string">"react"</span>;</span><br><span class="line"><span class="keyword">import</span> <span class="title class_">TodoItem</span> <span class="keyword">from</span> <span class="string">"./TodoItem"</span>;</span><br><span class="line"><span class="keyword">import</span> <span class="string">"./TodoList.css"</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> <span class="title function_">TodoList</span> = (<span class="params">{ todos, toggleComplete, deleteTodo }</span>) => {</span><br><span class="line"> <span class="keyword">if</span> (todos.<span class="property">length</span> === <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="language-xml"><span class="tag"><<span class="name">p</span> <span class="attr">className</span>=<span class="string">"empty-msg"</span>></span>No todos yet. Add something!<span class="tag"></<span class="name">p</span>></span></span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> (</span><br><span class="line"> <span class="language-xml"><span class="tag"><<span class="name">div</span> <span class="attr">className</span>=<span class="string">"todo-list"</span>></span></span></span><br><span class="line"><span class="language-xml"> {todos.map((todo) => (</span></span><br><span class="line"><span class="language-xml"> <span class="tag"><<span class="name">TodoItem</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">key</span>=<span class="string">{todo.id}</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">todo</span>=<span class="string">{todo}</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">toggleComplete</span>=<span class="string">{toggleComplete}</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> <span class="attr">deleteTodo</span>=<span class="string">{deleteTodo}</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml"> /></span></span></span><br><span class="line"><span class="language-xml"> ))}</span></span><br><span class="line"><span class="language-xml"> <span class="tag"></<span class="name">div</span>></span></span></span><br><span class="line"> );</span><br><span class="line">};</span><br><span class="line"></span><br><span class="line"><span class="keyword">export</span> <span class="keyword">default</span> <span class="title class_">TodoList</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<hr>
|
|
<h2 id="6️⃣-src-index-css（可选，简单全局样式）"><a href="#6️⃣-src-index-css（可选，简单全局样式）" class="headerlink" title="6️⃣ src/index.css（可选，简单全局样式）"></a>6️⃣ <code>src/index.css</code>（可选，简单全局样式）</h2><figure class="highlight css"><table><tr><td class="code"><pre><span class="line"><span class="comment">/* 让页面充满屏幕并居中 */</span></span><br><span class="line"><span class="selector-tag">html</span>, <span class="selector-tag">body</span>, <span class="selector-id">#root</span> {</span><br><span class="line"> <span class="attribute">height</span>: <span class="number">100%</span>;</span><br><span class="line"> <span class="attribute">margin</span>: <span class="number">0</span>;</span><br><span class="line"> <span class="attribute">font-family</span>: system-ui, sans-serif;</span><br><span class="line">}</span><br><span class="line"><span class="selector-tag">body</span> {</span><br><span class="line"> <span class="attribute">display</span>: flex;</span><br><span class="line"> <span class="attribute">justify-content</span>: center;</span><br><span class="line"> <span class="attribute">align-items</span>: flex-start;</span><br><span class="line"> <span class="attribute">background</span>: <span class="number">#f5f5f5</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<hr>
|
|
<h2 id="7️⃣-src-App-css"><a href="#7️⃣-src-App-css" class="headerlink" title="7️⃣ src/App.css"></a>7️⃣ <code>src/App.css</code></h2><figure class="highlight css"><table><tr><td class="code"><pre><span class="line"><span class="selector-class">.app-container</span> {</span><br><span class="line"> <span class="attribute">width</span>: <span class="number">90%</span>;</span><br><span class="line"> <span class="attribute">max-width</span>: <span class="number">480px</span>;</span><br><span class="line"> <span class="attribute">background</span>: <span class="number">#fff</span>;</span><br><span class="line"> <span class="attribute">padding</span>: <span class="number">20px</span>;</span><br><span class="line"> <span class="attribute">margin-top</span>: <span class="number">40px</span>;</span><br><span class="line"> <span class="attribute">box-shadow</span>: <span class="number">0</span> <span class="number">2px</span> <span class="number">10px</span> <span class="built_in">rgba</span>(<span class="number">0</span>,<span class="number">0</span>,<span class="number">0</span>,<span class="number">0.1</span>);</span><br><span class="line"> <span class="attribute">border-radius</span>: <span class="number">8px</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="selector-class">.app-container</span> <span class="selector-tag">h1</span> {</span><br><span class="line"> <span class="attribute">text-align</span>: center;</span><br><span class="line"> <span class="attribute">margin-bottom</span>: <span class="number">20px</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<hr>
|
|
<h2 id="8️⃣-src-components-TodoInput-css"><a href="#8️⃣-src-components-TodoInput-css" class="headerlink" title="8️⃣ src/components/TodoInput.css"></a>8️⃣ <code>src/components/TodoInput.css</code></h2><figure class="highlight css"><table><tr><td class="code"><pre><span class="line"><span class="selector-class">.todo-input</span> {</span><br><span class="line"> <span class="attribute">display</span>: flex;</span><br><span class="line"> <span class="attribute">margin-bottom</span>: <span class="number">20px</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="selector-class">.todo-input</span> <span class="selector-tag">input</span> {</span><br><span class="line"> <span class="attribute">flex</span>: <span class="number">1</span>;</span><br><span class="line"> <span class="attribute">padding</span>: <span class="number">8px</span> <span class="number">12px</span>;</span><br><span class="line"> <span class="attribute">font-size</span>: <span class="number">16px</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="selector-class">.todo-input</span> <span class="selector-tag">button</span> {</span><br><span class="line"> <span class="attribute">padding</span>: <span class="number">8px</span> <span class="number">12px</span>;</span><br><span class="line"> <span class="attribute">margin-left</span>: <span class="number">8px</span>;</span><br><span class="line"> <span class="attribute">font-size</span>: <span class="number">16px</span>;</span><br><span class="line"> <span class="attribute">background</span>: <span class="number">#4caf50</span>;</span><br><span class="line"> <span class="attribute">color</span>: <span class="number">#fff</span>;</span><br><span class="line"> <span class="attribute">border</span>: none;</span><br><span class="line"> <span class="attribute">cursor</span>: pointer;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="selector-class">.todo-input</span> <span class="selector-tag">button</span><span class="selector-pseudo">:hover</span> {</span><br><span class="line"> <span class="attribute">background</span>: <span class="number">#45a049</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<hr>
|
|
<h2 id="9️⃣-src-components-TodoList-css"><a href="#9️⃣-src-components-TodoList-css" class="headerlink" title="9️⃣ src/components/TodoList.css"></a>9️⃣ <code>src/components/TodoList.css</code></h2><figure class="highlight css"><table><tr><td class="code"><pre><span class="line"><span class="selector-class">.todo-list</span> {</span><br><span class="line"> <span class="attribute">display</span>: flex;</span><br><span class="line"> <span class="attribute">flex-direction</span>: column;</span><br><span class="line"> <span class="attribute">gap</span>: <span class="number">8px</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="selector-class">.empty-msg</span> {</span><br><span class="line"> <span class="attribute">text-align</span>: center;</span><br><span class="line"> <span class="attribute">color</span>: <span class="number">#777</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<hr>
|
|
<h2 id="🔟-src-components-TodoItem-css"><a href="#🔟-src-components-TodoItem-css" class="headerlink" title="🔟 src/components/TodoItem.css"></a>🔟 <code>src/components/TodoItem.css</code></h2><figure class="highlight css"><table><tr><td class="code"><pre><span class="line"><span class="selector-class">.todo-item</span> {</span><br><span class="line"> <span class="attribute">display</span>: flex;</span><br><span class="line"> <span class="attribute">align-items</span>: center;</span><br><span class="line"> <span class="attribute">justify-content</span>: space-between;</span><br><span class="line"> <span class="attribute">padding</span>: <span class="number">8px</span> <span class="number">12px</span>;</span><br><span class="line"> <span class="attribute">background</span>: <span class="number">#fafafa</span>;</span><br><span class="line"> <span class="attribute">border-radius</span>: <span class="number">4px</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="selector-class">.todo-item</span> <span class="selector-tag">label</span> {</span><br><span class="line"> <span class="attribute">flex</span>: <span class="number">1</span>;</span><br><span class="line"> <span class="attribute">display</span>: flex;</span><br><span class="line"> <span class="attribute">align-items</span>: center;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="selector-class">.todo-item</span> <span class="selector-tag">input</span><span class="selector-attr">[type=<span class="string">"checkbox"</span>]</span> {</span><br><span class="line"> <span class="attribute">margin-right</span>: <span class="number">12px</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="selector-class">.todo-item</span> <span class="selector-class">.completed</span> {</span><br><span class="line"> <span class="attribute">text-decoration</span>: line-through;</span><br><span class="line"> <span class="attribute">color</span>: <span class="number">#777</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="selector-class">.delete-btn</span> {</span><br><span class="line"> <span class="attribute">background</span>: transparent;</span><br><span class="line"> <span class="attribute">border</span>: none;</span><br><span class="line"> <span class="attribute">color</span>: <span class="number">#ff4444</span>;</span><br><span class="line"> <span class="attribute">font-size</span>: <span class="number">20px</span>;</span><br><span class="line"> <span class="attribute">cursor</span>: pointer;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="selector-class">.delete-btn</span><span class="selector-pseudo">:hover</span> {</span><br><span class="line"> <span class="attribute">color</span>: <span class="number">#ff0000</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<hr>
|
|
<h2 id="🎉-运行、打包"><a href="#🎉-运行、打包" class="headerlink" title="🎉 运行、打包"></a>🎉 运行、打包</h2><figure class="highlight bash"><table><tr><td class="code"><pre><span class="line"><span class="comment"># 开发模式</span></span><br><span class="line">npm run dev</span><br><span class="line"></span><br><span class="line"><span class="comment"># 生产构建</span></span><br><span class="line">npm run build</span><br></pre></td></tr></table></figure>
|
|
|
|
<hr>
|
|
<h2 id="🚀-小贴士"><a href="#🚀-小贴士" class="headerlink" title="🚀 小贴士"></a>🚀 小贴士</h2><ul>
|
|
<li><strong>本地存储</strong>：示例中使用 <code>localStorage</code> 让 Todo 在刷新后保留。生产环境可以接<br>入后端或使用 <code>IndexedDB</code> 之类的更强大存储方案。</li>
|
|
<li><strong>状态管理</strong>：若项目日后变大，可考虑引入<br><code>Redux</code>/<code>zustand</code>/<code>@tanstack/react-store</code> 等。</li>
|
|
<li><strong>类型安全</strong>：使用 <code>TypeScript</code> 可以防止类型错误；把文件后缀改成 <code>.tsx</code> 并在<br> <code>package.json</code> <code>dependencies</code> 增加 <code>typescript</code> 与 <code>@types/react</code>。</li>
|
|
<li><strong>单位测试</strong>：<code>jest</code>、<code>react-testing-library</code> 能帮你保障组件逻辑正确。</li>
|
|
<li><strong>样式优化</strong>：考虑使用 <code>styled-components</code>/<code>emotion</code>/<code>tailwindcss</code> 等，让样<br>式变得更易维护。</li>
|
|
</ul>
|
|
<p>现在你就拥有一个完整、可直接跑起的 Todo React 应用，随时可以继续扩展功能（如过<br>滤、拖拽排序、用户登录等）。祝编码愉快 🚀！</p>
|
|
<hr>
|
|
<p>在本地先建个应用</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">npm create vite@latest todo-app --template react</span><br></pre></td></tr></table></figure>
|
|
<p>这里要注意下，要用vite的话要node版本在<code>20.19+</code>或者<code>22.12+</code><br>还有是需要选择js版本，而不是ts<br>运行下</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">npm install</span><br></pre></td></tr></table></figure>
|
|
<p>这里比较慢的话可以上镜像</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">npm config set registry https://registry.npmmirror.com</span><br></pre></td></tr></table></figure>
|
|
<p>接着把生成的代码都保存成文件</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">├── src</span><br><span class="line">│ ├── App.css</span><br><span class="line">│ ├── App.jsx</span><br><span class="line">│ ├── assets</span><br><span class="line">│ │ └── react.svg</span><br><span class="line">│ ├── components</span><br><span class="line">│ │ ├── TodoInput.css</span><br><span class="line">│ │ ├── TodoInput.jsx</span><br><span class="line">│ │ ├── TodoItem.css</span><br><span class="line">│ │ ├── TodoItem.jsx</span><br><span class="line">│ │ ├── TodoList.css</span><br><span class="line">│ │ └── TodoList.jsx</span><br><span class="line">│ ├── index.css</span><br><span class="line">│ └── main.jsx</span><br></pre></td></tr></table></figure>
|
|
<p>运行命令</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">npm run dev</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>看到效果<br><img data-src="https://img.nicksxs.me/gpt-oss-todo.png"></p>
|
|
<p>结果是真的直接可用的，为什么要测这个呢，主要原因是一个模型本地可用的探讨，我认为如果是要本地可用，至少在这些相对简单的任务上能够完成得比较好，前面那个只是个提问，并没有什么可以验证正确性的，内容还是比较多的，对于做开发的来说能辅助编程还是比较重要的，并且能够完成一些相对独立完整的任务才是比较可用的初级门槛，只是对内存压力还是比较大的</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>给小电驴上牌</title>
|
|
<url>/2022/03/20/%E7%BB%99%E5%B0%8F%E7%94%B5%E9%A9%B4%E4%B8%8A%E7%89%8C/</url>
|
|
<content><![CDATA[<p>三八节活动的时候下决心买了个小电驴，主要是上下班路上现在通勤条件越来越恶劣了，之前都是觉得坐公交就行了，实际路程就比较短，但是现在或者说大概是年前那两个月差不多就开始了，基本是堵一路，个人感觉是天目山路那边在修地铁，而且蚂蚁的几个空间都在那，上班的时间点都差不多，前一个修地铁感觉挺久了，机动车保有量也越来越多，总体是古墩路就越来越堵，还有个原因就是早上上班的点共享单车都被骑走了，有时候整整走一路都没一辆，有时候孤零零地有一辆基本都是破的；走路其实也是一种选择，但是因为要赶着上班，走得太慢就要很久，可能要 45 分钟这样，走得比较快就一身汗挺难受的。所以考虑自行车和电动车，这里还有一点就是不管是乘公交还是骑共享单车，其实都要从楼下走出去蛮远，公司回来也是，也就是这种通勤方式在准备阶段就花了比较多时间，比如总的从下班到到家的时间是半小时，可能在骑共享单车和公交车上的时间都不到十分钟，就比较难受。觉得这种比例太浪费时间，如果能有这种比较点对点的方式，估计能省时省力不少，前面说的骑共享单车的方式其实在之前是比较可行的，但是后来越来越少车，基本都是每周的前几天，周一到周三都是没有车，走路到公司再冷的天都是走出一身的汗，下雨天就更难受，本来下雨天应该是优先选择坐公交，但是一般下雨天堵车会更严重，而且车子到我上车的那个站，下雨天就挤得不行，总体说下来感觉事情都不打，但是几年下来，还是会挺不爽的。</p>
|
|
<p>电驴看的比较草率，主要是考虑续航，然后锂电池外加 48v 和 24AH，这样一般来讲还是价格比较高的，只是原来没预料到这个限速，以为现在的车子都比较快，但是现在的新国标车子都是 25km/h 的限速，然后 15km/h 都是会要提醒，虽然说有一些特殊的解除限速的方法，但是解了也就 35km/h ，差距不是特别大，而且现在的车子都是比较小，也不太能载东西，特别是上下班路程也不远的情况下，其实不是那么需要速度，就像我朋友说的，可能骑车的时间还不如等红绿灯多，所以就还好，也不打算解除限速，只是品牌上也仔细看，后来选了绿源，目前大部分还是雅迪，爱玛，台羚，绿源，小牛等，路上看的话还是雅迪比较多，不过价格也比较贵一点，还有就是小牛了，是比较新兴的品牌，手机 App 什么的做得比较好，而且也比较贵，最后以相对比较便宜的价格买了个锂电 48V24AH 的小车子，后来发现还是有点不方便的点就是没有比较大的筐，也不好装，这样就是下雨天雨衣什么的比较不方便放。</p>
|
|
<p>聊回来主题上牌这个事情，这个事情也是颇费心力，提车的时候店里的让我跟他早上一起去，但是因为不确定时间，也比较远就没跟着去，因为我是线上买的，线下自提，线下的店可能没啥利润可以拿，就不肯帮忙代上牌，朋友说在线下店里买是可以代上的，自己上牌过程也比较曲折，一开始是头盔没到，然后是等开发票，主要的东西就是需要骑着车子去车管所，不能只自己去，然后需要预约，附近比较近的都是提前一周就预约完了号了，要提前在支付宝上进行预约，比较空的就是店里推荐的景区大队，但是随之而来就是比较蛋疼的，这个景区大队太远了，看下骑车距离有十几公里，所以就有点拖延症，但是总归要上的，不然一直不能开是白买了，上牌的材料主要是车辆合格证，发票，然后车子上的浙品码，在车架上和电池上，然后车架号什么的都要跟合格证上完全对应，整体车子要跟合格证上一毛一样，如果有额外的反光镜，后面副座都需要拆掉，脚踏板要装上，到了那其实还比较顺利，就是十几公里外加那天比较冷，吹得头疼。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>真正适合spring ai的使用方式</title>
|
|
<url>/2025/08/24/%E7%9C%9F%E6%AD%A3%E9%80%82%E5%90%88spring-ai%E7%9A%84%E4%BD%BF%E7%94%A8%E6%96%B9%E5%BC%8F/</url>
|
|
<content><![CDATA[<p>上周被spring-ai调用模型api这个折腾的有点难受，这周又试了下用deepseek官方的api，但结果还是一样，不确定是啥原因，请求会一直卡着<br>后来想着本地有ollama，那是不是可以直接用ollama来调本地模型<br>这里我们用一个非常基础很小的模型</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">ollama run gemma3:1b</span><br></pre></td></tr></table></figure>
|
|
<p>然后对应的配置是</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.ai<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-ai-ollama-spring-boot-starter<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"><span class="tag"></<span class="name">dependency</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>当然这里有个坑，因为目前还不成熟，所以这个包的依赖跟之前openai的是不兼容的<br>需要引用</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.ai<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-ai-bom<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>1.0.0-M1<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">type</span>></span>pom<span class="tag"></<span class="name">type</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">scope</span>></span>import<span class="tag"></<span class="name">scope</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>属性配置就很简单了</p>
|
|
<figure class="highlight properties"><table><tr><td class="code"><pre><span class="line"><span class="attr">spring.ai.ollama.base-url</span>=<span class="string">http://localhost:11434</span></span><br><span class="line"><span class="attr">spring.ai.ollama.chat.model</span>=<span class="string">gemma3:1b</span></span><br></pre></td></tr></table></figure>
|
|
<p>然后就是简单写个Controller就可以了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@RestController</span></span><br><span class="line"><span class="meta">@RequestMapping("/demo")</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">OllamaController</span> {</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">Logger</span> <span class="variable">logger</span> <span class="operator">=</span> LoggerFactory.getLogger(OllamaController.class);</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Autowired</span></span><br><span class="line"> <span class="keyword">private</span> OllamaChatModel ollamaChatModel;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="meta">@RequestMapping("/ollama")</span></span><br><span class="line"> <span class="keyword">public</span> Object <span class="title function_">ollama</span><span class="params">(String msg)</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">call</span> <span class="operator">=</span> ollamaChatModel.call(msg);</span><br><span class="line"> <span class="keyword">return</span> call;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这样我们的初始调用就完成了<br>比如我们提问 ‘how to learn java’</p>
|
|
<figure class="highlight markdown"><table><tr><td class="code"><pre><span class="line">Okay, let's break down how to learn Java! It can seem daunting at first, but with a structured approach, you'll be coding in Java in no time. Here’s a breakdown of a recommended learning path, broken down into phases: <span class="strong">**1. Foundation (2-4 Weeks)**</span> * <span class="strong">**Understand the Basics:**</span> * <span class="strong">**What is Java?**</span> Java is a robust, object-oriented programming language known for its platform independence ("write once, run anywhere"). It’s used for enterprise applications, Android development, and more. * <span class="strong">**Why Learn Java?**</span> It’s a popular choice for beginners because it’s relatively easy to learn, has a large community, and is used in many real-world applications. * <span class="strong">**Key Concepts:**</span> * <span class="strong">**Variables & Data Types:**</span> Understand different data types (int, float, String, boolean, etc.). * <span class="strong">**Operators:**</span> Arithmetic, comparison, logical, assignment. * <span class="strong">**Control Flow:**</span> <span class="code">`if`</span>, <span class="code">`else`</span>, <span class="code">`for`</span>, <span class="code">`while`</span> – controlling the flow of your program. * <span class="strong">**Basic Syntax:**</span> The code structure of Java. * <span class="strong">**Resources:**</span> * <span class="strong">**Tutorialspoint Java Course:**</span> [<span class="string">https://www.tutorialspoint.com/java/index.htm</span>](<span class="link">https://www.tutorialspoint.com/java/index.htm</span>) - A good, comprehensive, and free overview. * <span class="strong">**Codecademy's Learn Java:**</span> [<span class="string">https://www.codecademy.com/learn/learn-java</span>](<span class="link">https://www.codecademy.com/learn/learn-java</span>) - Interactive, hands-on exercises. * <span class="strong">**Google's Java Tutorial:**</span> [<span class="string">https://developer.android.com/training/jdbc</span>](<span class="link">https://developer.android.com/training/jdbc</span>) (Java is built on JDBC, so this will give you a baseline) <span class="strong">**2. Core Concepts (4-8 Weeks)**</span> * <span class="strong">**Data Structures:**</span> * <span class="strong">**Arrays:**</span> Fundamental data structures. Understand how to declare, initialize, and use arrays. * <span class="strong">**Linked Lists:**</span> Another essential data structure, offering flexibility. * <span class="strong">**Object-Oriented Programming (OOP) Fundamentals:**</span> * <span class="strong">**Classes and Objects:**</span> The basis of OOP. Learn how to define classes, create objects, and interact with them. * <span class="strong">**Encapsulation:**</span> Bundling data and methods that operate on that data within a class. * <span class="strong">**Inheritance:**</span> Creating new classes based on existing classes. * <span class="strong">**Polymorphism:**</span> The ability of objects to take on many forms. * <span class="strong">**Java Syntax & Semantics:**</span> * <span class="strong">**Statements:**</span> The commands you write. * <span class="strong">**Expressions:**</span> Results of calculations. * <span class="strong">**Keywords:**</span> Reserved words with special meanings (e.g., <span class="code">`class`</span>, <span class="code">`if`</span>, <span class="code">`else`</span>). * <span class="strong">**Comments:**</span> Explain your code. * <span class="strong">**Resources:**</span> * <span class="strong">**Oracle Java Tutorials:**</span> [<span class="string">https://docs.oracle.com/javase/tutorial/</span>](<span class="link">https://docs.oracle.com/javase/tutorial/</span>) – A more in-depth guide. * <span class="strong">**FreeCodeCamp's Java Tutorial:**</span> [<span class="string">https://www.freecodecamp.org/learn/java-tutorial/</span>](<span class="link">https://www.freecodecamp.org/learn/java-tutorial/</span>) - Great for visual learners. <span class="strong">**3. Practical Application - Projects**</span> * <span class="strong">**Simple Programs:**</span> Start with small, manageable projects: * <span class="strong">**Number Guessing Game:**</span> A classic introductory project. * <span class="strong">**Rock-Paper-Scissors:**</span> A basic example of OOP. * <span class="strong">**Simple Calculator:**</span> Reinforces fundamental concepts. * <span class="strong">**Gradually Increase Complexity:**</span> * <span class="strong">**Print Statements:**</span> A simple way to understand output. * <span class="strong">**Palindrome Checker:**</span> A challenge to practice string manipulation. * <span class="strong">**Basic Console Applications:**</span> Write programs that run directly in the console. <span class="strong">**4. Intermediate Concepts (Ongoing)**</span> * <span class="strong">**Exception Handling:**</span> Understanding and handling errors. * <span class="strong">**Collections:**</span> Learn about ArrayLists, LinkedLists, Sets, and Maps. * <span class="strong">**String Manipulation:**</span> Working with strings effectively. * <span class="strong">**Multithreading:**</span> Understanding how to run multiple tasks concurrently (basic concepts). * <span class="strong">**Java Collections Framework:**</span> The cornerstone of modern Java programming. <span class="strong">**Resources for Continued Learning:**</span> * <span class="strong">**Stack Overflow:**</span> [<span class="string">https://stackoverflow.com/</span>](<span class="link">https://stackoverflow.com/</span>) - An invaluable resource for solving coding problems and learning from others. * <span class="strong">**Reddit's r/java:**</span> [<span class="string">https://www.reddit.com/r/java/</span>](<span class="link">https://www.reddit.com/r/java/</span>) - A community forum with discussions and resources. * <span class="strong">**YouTube:**</span> Search for Java tutorials and courses (e.g., "Java for beginners"). <span class="strong">**Tips for Success:**</span> * <span class="strong">**Practice Regularly:**</span> Even 30 minutes a day is better than long, infrequent sessions. * <span class="strong">**Write Code Frequently:**</span> The more you write, the better you'll understand. * <span class="strong">**Don't Be Afraid to Make Mistakes:**</span> Mistakes are part of the learning process. Learn from them. * <span class="strong">**Read Code:**</span> Look at code written by others to understand different approaches. * <span class="strong">**Join a Community:**</span> Connecting with others can provide support and motivation. <span class="strong">**To help me tailor my advice even more, could you tell me:**</span> * <span class="strong">**What's your background in programming?**</span> (e.g., Have you programmed before? If so, what languages?) * <span class="strong">**What are your goals for learning Java?**</span> (e.g., Career change? Personal project? Specific application like Android?)</span><br></pre></td></tr></table></figure>
|
|
<p>格式不太美观，但是至少是能输出来结果了，而且可以选择本地可用的最佳模型，话说openai的oss试了下，本地有点吃不消，不然可能是比较合适的</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊一下 Java 的日志系列一</title>
|
|
<url>/2024/01/07/%E8%81%8A%E4%B8%80%E4%B8%8B-Java-%E7%9A%84%E6%97%A5%E5%BF%97%E7%B3%BB%E5%88%97%E4%B8%80/</url>
|
|
<content><![CDATA[<p>我们在使用 Java 的日志库的时候，比如我们现在项目在用的 logback，可以配置滚动策略，简单介绍下启动逻辑,这里我们定义的是<br><code>ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy</code> 它是继承了 <code>ch.qos.logback.core.rolling.TimeBasedRollingPolicy</code><br>它的启动方法就是下面这个 <code>start</code> 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">start</span><span class="params">()</span> {</span><br><span class="line"> SizeAndTimeBasedFNATP<E> sizeAndTimeBasedFNATP = <span class="keyword">new</span> <span class="title class_">SizeAndTimeBasedFNATP</span><E>(Usage.EMBEDDED); </span><br><span class="line"> <span class="keyword">if</span>(maxFileSize == <span class="literal">null</span>) {</span><br><span class="line"> addError(<span class="string">"maxFileSize property is mandatory."</span>);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> addInfo(<span class="string">"Archive files will be limited to ["</span>+maxFileSize+<span class="string">"] each."</span>);</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> sizeAndTimeBasedFNATP.setMaxFileSize(maxFileSize);</span><br><span class="line"> timeBasedFileNamingAndTriggeringPolicy = sizeAndTimeBasedFNATP;</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">if</span>(!isUnboundedTotalSizeCap() && totalSizeCap.getSize() < maxFileSize.getSize()) {</span><br><span class="line"> addError(<span class="string">"totalSizeCap of ["</span>+totalSizeCap+<span class="string">"] is smaller than maxFileSize ["</span>+maxFileSize+<span class="string">"] which is non-sensical"</span>);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="comment">// most work is done by the parent</span></span><br><span class="line"> <span class="built_in">super</span>.start();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>配置了 timeBasedFileNamingAndTriggeringPolicy 策略</p>
|
|
<p>然后调用了父类的启动方法，主要看下父类的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">start</span><span class="params">()</span> {</span><br><span class="line"> <span class="comment">// set the LR for our utility object</span></span><br><span class="line"> renameUtil.setContext(<span class="built_in">this</span>.context);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// find out period from the filename pattern</span></span><br><span class="line"> <span class="keyword">if</span> (fileNamePatternStr != <span class="literal">null</span>) {</span><br><span class="line"> fileNamePattern = <span class="keyword">new</span> <span class="title class_">FileNamePattern</span>(fileNamePatternStr, <span class="built_in">this</span>.context);</span><br><span class="line"> determineCompressionMode();</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> addWarn(FNP_NOT_SET);</span><br><span class="line"> addWarn(CoreConstants.SEE_FNP_NOT_SET);</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalStateException</span>(FNP_NOT_SET + CoreConstants.SEE_FNP_NOT_SET);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> compressor = <span class="keyword">new</span> <span class="title class_">Compressor</span>(compressionMode);</span><br><span class="line"> compressor.setContext(context);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// wcs : without compression suffix</span></span><br><span class="line"> fileNamePatternWithoutCompSuffix = <span class="keyword">new</span> <span class="title class_">FileNamePattern</span>(Compressor.computeFileNameStrWithoutCompSuffix(fileNamePatternStr, compressionMode), <span class="built_in">this</span>.context);</span><br><span class="line"></span><br><span class="line"> addInfo(<span class="string">"Will use the pattern "</span> + fileNamePatternWithoutCompSuffix + <span class="string">" for the active file"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (compressionMode == CompressionMode.ZIP) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">zipEntryFileNamePatternStr</span> <span class="operator">=</span> transformFileNamePattern2ZipEntry(fileNamePatternStr);</span><br><span class="line"> zipEntryFileNamePattern = <span class="keyword">new</span> <span class="title class_">FileNamePattern</span>(zipEntryFileNamePatternStr, context);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (timeBasedFileNamingAndTriggeringPolicy == <span class="literal">null</span>) {</span><br><span class="line"> timeBasedFileNamingAndTriggeringPolicy = <span class="keyword">new</span> <span class="title class_">DefaultTimeBasedFileNamingAndTriggeringPolicy</span><E>();</span><br><span class="line"> }</span><br><span class="line"> timeBasedFileNamingAndTriggeringPolicy.setContext(context);</span><br><span class="line"> timeBasedFileNamingAndTriggeringPolicy.setTimeBasedRollingPolicy(<span class="built_in">this</span>);</span><br><span class="line"> timeBasedFileNamingAndTriggeringPolicy.start();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!timeBasedFileNamingAndTriggeringPolicy.isStarted()) {</span><br><span class="line"> addWarn(<span class="string">"Subcomponent did not start. TimeBasedRollingPolicy will not start."</span>);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// the maxHistory property is given to TimeBasedRollingPolicy instead of to</span></span><br><span class="line"> <span class="comment">// the TimeBasedFileNamingAndTriggeringPolicy. This makes it more convenient</span></span><br><span class="line"> <span class="comment">// for the user at the cost of inconsistency here.</span></span><br><span class="line"> <span class="keyword">if</span> (maxHistory != UNBOUND_HISTORY) {</span><br><span class="line"> archiveRemover = timeBasedFileNamingAndTriggeringPolicy.getArchiveRemover();</span><br><span class="line"> archiveRemover.setMaxHistory(maxHistory);</span><br><span class="line"> archiveRemover.setTotalSizeCap(totalSizeCap.getSize());</span><br><span class="line"> <span class="keyword">if</span> (cleanHistoryOnStart) {</span><br><span class="line"> addInfo(<span class="string">"Cleaning on start up"</span>);</span><br><span class="line"> <span class="type">Date</span> <span class="variable">now</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Date</span>(timeBasedFileNamingAndTriggeringPolicy.getCurrentTime());</span><br><span class="line"> cleanUpFuture = archiveRemover.cleanAsynchronously(now);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (!isUnboundedTotalSizeCap()) {</span><br><span class="line"> addWarn(<span class="string">"'maxHistory' is not set, ignoring 'totalSizeCap' option with value ["</span>+totalSizeCap+<span class="string">"]"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">super</span>.start();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<ul>
|
|
<li><p>第一步是给 renameUtil 设置 context</p>
|
|
</li>
|
|
<li><p>第二步是判断 fileNamePatternStr是否已配置，没配置会报错，如果配置了就会去判断压缩格式，是否要压缩以及压缩的是 gz 还是 zip</p>
|
|
</li>
|
|
<li><p>第三步就是设置压缩器了</p>
|
|
</li>
|
|
<li><p>第四步是判断文件后缀格式，注意是不带压缩格式的</p>
|
|
</li>
|
|
<li><p>第五步是判断 timeBasedFileNamingAndTriggeringPolicy 是否已设置，这里是在子类里已经设置了 <code>ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP</code> 否则就是 <code>ch.qos.logback.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy</code> 这个默认的</p>
|
|
</li>
|
|
<li><p>第六步比较重要就是启动 <code>timeBasedFileNamingAndTriggeringPolicy</code></p>
|
|
</li>
|
|
</ul>
|
|
<p><code>timeBasedFileNamingAndTriggeringPolicy</code> 的启动逻辑里首先是调用了 <code>SizeAndTimeBasedFNATP</code> 的 start 方法，然后里面最开始调用了父类的 start</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">start</span><span class="params">()</span> {</span><br><span class="line"> <span class="comment">// we depend on certain fields having been initialized in super class</span></span><br><span class="line"> <span class="built_in">super</span>.start();</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">if</span>(usage == Usage.DIRECT) {</span><br><span class="line"> addWarn(CoreConstants.SIZE_AND_TIME_BASED_FNATP_IS_DEPRECATED);</span><br><span class="line"> addWarn(<span class="string">"For more information see "</span>+MANUAL_URL_PREFIX+<span class="string">"appenders.html#SizeAndTimeBasedRollingPolicy"</span>);</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">if</span> (!<span class="built_in">super</span>.isErrorFree())</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"></span><br><span class="line"> </span><br><span class="line"> <span class="keyword">if</span> (maxFileSize == <span class="literal">null</span>) {</span><br><span class="line"> addError(<span class="string">"maxFileSize property is mandatory."</span>);</span><br><span class="line"> withErrors();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!validateDateAndIntegerTokens()) {</span><br><span class="line"> withErrors();</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> archiveRemover = createArchiveRemover();</span><br><span class="line"> archiveRemover.setContext(context);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// we need to get the correct value of currentPeriodsCounter.</span></span><br><span class="line"> <span class="comment">// usually the value is 0, unless the appender or the application</span></span><br><span class="line"> <span class="comment">// is stopped and restarted within the same period</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">regex</span> <span class="operator">=</span> tbrp.fileNamePattern.toRegexForFixedDate(dateInCurrentPeriod);</span><br><span class="line"> <span class="type">String</span> <span class="variable">stemRegex</span> <span class="operator">=</span> FileFilterUtil.afterLastSlash(regex);</span><br><span class="line"></span><br><span class="line"> computeCurrentPeriodsHighestCounterValue(stemRegex);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (isErrorFree()) {</span><br><span class="line"> started = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>也就是下面的代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">start</span><span class="params">()</span> {</span><br><span class="line"> DateTokenConverter<Object> dtc = tbrp.fileNamePattern.getPrimaryDateTokenConverter();</span><br><span class="line"> <span class="keyword">if</span> (dtc == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalStateException</span>(<span class="string">"FileNamePattern ["</span> + tbrp.fileNamePattern.getPattern() + <span class="string">"] does not contain a valid DateToken"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (dtc.getTimeZone() != <span class="literal">null</span>) {</span><br><span class="line"> rc = <span class="keyword">new</span> <span class="title class_">RollingCalendar</span>(dtc.getDatePattern(), dtc.getTimeZone(), Locale.getDefault());</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> rc = <span class="keyword">new</span> <span class="title class_">RollingCalendar</span>(dtc.getDatePattern());</span><br><span class="line"> }</span><br><span class="line"> addInfo(<span class="string">"The date pattern is '"</span> + dtc.getDatePattern() + <span class="string">"' from file name pattern '"</span> + tbrp.fileNamePattern.getPattern() + <span class="string">"'."</span>);</span><br><span class="line"> rc.printPeriodicity(<span class="built_in">this</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!rc.isCollisionFree()) {</span><br><span class="line"> addError(<span class="string">"The date format in FileNamePattern will result in collisions in the names of archived log files."</span>);</span><br><span class="line"> addError(CoreConstants.MORE_INFO_PREFIX + COLLIDING_DATE_FORMAT_URL);</span><br><span class="line"> withErrors();</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> setDateInCurrentPeriod(<span class="keyword">new</span> <span class="title class_">Date</span>(getCurrentTime()));</span><br><span class="line"> <span class="keyword">if</span> (tbrp.getParentsRawFileProperty() != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">File</span> <span class="variable">currentFile</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">File</span>(tbrp.getParentsRawFileProperty());</span><br><span class="line"> <span class="keyword">if</span> (currentFile.exists() && currentFile.canRead()) {</span><br><span class="line"> setDateInCurrentPeriod(<span class="keyword">new</span> <span class="title class_">Date</span>(currentFile.lastModified()));</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> addInfo(<span class="string">"Setting initial period to "</span> + dateInCurrentPeriod);</span><br><span class="line"> computeNextCheck();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>前面是日期时区等配置处理，然后是判断按日期生成的文件会不会冲突，接下去是设置当前时间段，如果配置了日志文件的话就会把当前时间段设置成已有的日志文件的最后更改时间，最后的 <code>computeNextCheck</code> 比较重要</p>
|
|
<p>主要是下面的方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">computeNextCheck</span><span class="params">()</span> {</span><br><span class="line"> nextCheck = rc.getNextTriggeringDate(dateInCurrentPeriod).getTime();</span><br><span class="line"> }</span><br><span class="line"><span class="keyword">public</span> Date <span class="title function_">getNextTriggeringDate</span><span class="params">(Date now)</span> {</span><br><span class="line"> <span class="keyword">return</span> getEndOfNextNthPeriod(now, <span class="number">1</span>);</span><br><span class="line"> }</span><br><span class="line"><span class="keyword">public</span> Date <span class="title function_">getEndOfNextNthPeriod</span><span class="params">(Date now, <span class="type">int</span> periods)</span> {</span><br><span class="line"> <span class="keyword">return</span> innerGetEndOfNextNthPeriod(<span class="built_in">this</span>, <span class="built_in">this</span>.periodicityType, now, periods);</span><br><span class="line"> }</span><br><span class="line"><span class="keyword">static</span> <span class="keyword">private</span> Date <span class="title function_">innerGetEndOfNextNthPeriod</span><span class="params">(Calendar cal, PeriodicityType periodicityType, Date now, <span class="type">int</span> numPeriods)</span> {</span><br><span class="line"> cal.setTime(now);</span><br><span class="line"> <span class="keyword">switch</span> (periodicityType) {</span><br><span class="line"> <span class="keyword">case</span> TOP_OF_MILLISECOND:</span><br><span class="line"> cal.add(Calendar.MILLISECOND, numPeriods);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">case</span> TOP_OF_SECOND:</span><br><span class="line"> cal.set(Calendar.MILLISECOND, <span class="number">0</span>);</span><br><span class="line"> cal.add(Calendar.SECOND, numPeriods);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">case</span> TOP_OF_MINUTE:</span><br><span class="line"> cal.set(Calendar.SECOND, <span class="number">0</span>);</span><br><span class="line"> cal.set(Calendar.MILLISECOND, <span class="number">0</span>);</span><br><span class="line"> cal.add(Calendar.MINUTE, numPeriods);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">case</span> TOP_OF_HOUR:</span><br><span class="line"> cal.set(Calendar.MINUTE, <span class="number">0</span>);</span><br><span class="line"> cal.set(Calendar.SECOND, <span class="number">0</span>);</span><br><span class="line"> cal.set(Calendar.MILLISECOND, <span class="number">0</span>);</span><br><span class="line"> cal.add(Calendar.HOUR_OF_DAY, numPeriods);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">case</span> TOP_OF_DAY:</span><br><span class="line"> cal.set(Calendar.HOUR_OF_DAY, <span class="number">0</span>);</span><br><span class="line"> cal.set(Calendar.MINUTE, <span class="number">0</span>);</span><br><span class="line"> cal.set(Calendar.SECOND, <span class="number">0</span>);</span><br><span class="line"> cal.set(Calendar.MILLISECOND, <span class="number">0</span>);</span><br><span class="line"> cal.add(Calendar.DATE, numPeriods);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">case</span> TOP_OF_WEEK:</span><br><span class="line"> cal.set(Calendar.DAY_OF_WEEK, cal.getFirstDayOfWeek());</span><br><span class="line"> cal.set(Calendar.HOUR_OF_DAY, <span class="number">0</span>);</span><br><span class="line"> cal.set(Calendar.MINUTE, <span class="number">0</span>);</span><br><span class="line"> cal.set(Calendar.SECOND, <span class="number">0</span>);</span><br><span class="line"> cal.set(Calendar.MILLISECOND, <span class="number">0</span>);</span><br><span class="line"> cal.add(Calendar.WEEK_OF_YEAR, numPeriods);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">case</span> TOP_OF_MONTH:</span><br><span class="line"> cal.set(Calendar.DATE, <span class="number">1</span>);</span><br><span class="line"> cal.set(Calendar.HOUR_OF_DAY, <span class="number">0</span>);</span><br><span class="line"> cal.set(Calendar.MINUTE, <span class="number">0</span>);</span><br><span class="line"> cal.set(Calendar.SECOND, <span class="number">0</span>);</span><br><span class="line"> cal.set(Calendar.MILLISECOND, <span class="number">0</span>);</span><br><span class="line"> cal.add(Calendar.MONTH, numPeriods);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">default</span>:</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalStateException</span>(<span class="string">"Unknown periodicity type."</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> cal.getTime();</span><br></pre></td></tr></table></figure>
|
|
<p> 会按照我们设置的 FileNamePattern 中的 datePattern 来推断 periodicityType，比如我们是小时滚动的，那就是 <code>TOP_OF_HOUR</code> ,算出来下一个时间检查点，后面会按这个判断是否作为触发事件触发日志滚动更新</p>
|
|
<p>父类的 start 逻辑讲完以后，子类的其实比较简单，先判断使用方式，我们这是嵌入式的，然后是父类有没有产生错误，继续是最大文件大小是否设置了，再判断日期格式是否正常，然后是归档移除类的创建，并设置到上下文中，然后计算当前时间的最大日志文件计数器，最后判断是否报错，没有的话就启动成功了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>简单介绍下mcp是什么</title>
|
|
<url>/2025/05/04/%E7%AE%80%E5%8D%95%E4%BB%8B%E7%BB%8D%E4%B8%8Bmcp%E6%98%AF%E4%BB%80%E4%B9%88/</url>
|
|
<content><![CDATA[<p>在大模型的演进过程中，mcp是个对于使用者非常有用的一个协议或者说工具，Model Context Protocol (MCP) 是一种专为大型语言模型和 AI 系统设计的通信协议框架，它解决了 AI 交互中的一个核心问题：如何有效地管理、传递和控制上下文信息。打个比方，如果把 AI 模型比作一个智能助手，那么 MCP 就是确保这个助手能够”记住”之前的对话、理解当前问题的背景，并按照特定规则进行回应的通信机制。<br>MCP 的工作原理<br>基本结构<br>MCP 的基本结构可以分为三个主要部分：</p>
|
|
<ol>
|
|
<li>上下文容器 (Context Container)：存储对话历史、系统指令和用户背景等信息</li>
|
|
<li>控制参数 (Control Parameters)：调节模型行为的设置，如温度、最大输出长度等</li>
|
|
<li>消息体 (Message Body)：当前需要处理的输入内容<br>┌─────────────────────────────────┐<br>│ MCP 请求/响应结构 │</li>
|
|
</ol>
|
|
<p>├─────────────────────────────────┤<br>│ │<br>│ ┌─────────────────────────┐ │<br>│ │ 上下文容器 │ │<br>│ │ ┌─────────────────┐ │ │<br>│ │ │ 对话历史 │ │ │<br>│ │ └─────────────────┘ │ │<br>│ │ ┌─────────────────┐ │ │<br>│ │ │ 系统指令 │ │ │<br>│ │ └─────────────────┘ │ │<br>│ │ ┌─────────────────┐ │ │<br>│ │ │ 用户背景 │ │ │<br>│ │ └─────────────────┘ │ │<br>│ └─────────────────────────┘ │<br>│ │<br>│ ┌─────────────────────────┐ │<br>│ │ 控制参数 │ │<br>│ └─────────────────────────┘ │<br>│ │<br>│ ┌─────────────────────────┐ │<br>│ │ 消息体 │ │<br>│ └─────────────────────────┘ │<br>│ │<br>└─────────────────────────────────┘</p>
|
|
<h2 id="实操"><a href="#实操" class="headerlink" title="实操"></a>实操</h2><p>首先基于start.spring.io创建一个springboot应用，需要springboot的版本3.3+和jdk17+<br>然后添加maven依赖</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.ai<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-ai-starter-mcp-server<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>1.0.0-M8<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-web<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>6.1.12<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>我们就可以实现一个mcp的demo server</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">package</span> com.nicksxs.mcp_demo;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> com.jayway.jsonpath.JsonPath;</span><br><span class="line"><span class="keyword">import</span> org.springframework.ai.tool.annotation.Tool;</span><br><span class="line"><span class="keyword">import</span> org.springframework.ai.tool.annotation.ToolParam;</span><br><span class="line"><span class="keyword">import</span> org.springframework.stereotype.Service;</span><br><span class="line"><span class="keyword">import</span> org.springframework.web.client.RestClient;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> java.util.List;</span><br><span class="line"><span class="keyword">import</span> java.util.Map;</span><br><span class="line"></span><br><span class="line"><span class="meta">@Service</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">WeatherService</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> RestClient restClient;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="title function_">WeatherService</span><span class="params">()</span> {</span><br><span class="line"> <span class="built_in">this</span>.restClient = RestClient.builder()</span><br><span class="line"> .baseUrl(<span class="string">"https://api.weather.gov"</span>)</span><br><span class="line"> .defaultHeader(<span class="string">"Accept"</span>, <span class="string">"application/geo+json"</span>)</span><br><span class="line"> .defaultHeader(<span class="string">"User-Agent"</span>, <span class="string">"WeatherApiClient/1.0 (your@email.com)"</span>)</span><br><span class="line"> .build();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Tool(description = "Get weather forecast for a specific latitude/longitude")</span></span><br><span class="line"> <span class="keyword">public</span> String <span class="title function_">getWeatherForecastByLocation</span><span class="params">(</span></span><br><span class="line"><span class="params"> <span class="type">double</span> latitude, // Latitude coordinate</span></span><br><span class="line"><span class="params"> <span class="type">double</span> longitude // Longitude coordinate</span></span><br><span class="line"><span class="params"> )</span> {</span><br><span class="line"> <span class="comment">// 首先获取点位信息</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">pointsResponse</span> <span class="operator">=</span> restClient.get()</span><br><span class="line"> .uri(<span class="string">"/points/{lat},{lon}"</span>, latitude, longitude)</span><br><span class="line"> .retrieve()</span><br><span class="line"> .body(String.class);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 从点位响应中提取预报URL</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">forecastUrl</span> <span class="operator">=</span> JsonPath.read(pointsResponse, <span class="string">"$.properties.forecast"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 获取天气预报</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">forecast</span> <span class="operator">=</span> restClient.get()</span><br><span class="line"> .uri(forecastUrl)</span><br><span class="line"> .retrieve()</span><br><span class="line"> .body(String.class);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 从预报中提取第一个周期的详细信息</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">detailedForecast</span> <span class="operator">=</span> JsonPath.read(forecast, <span class="string">"$.properties.periods[0].detailedForecast"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">temperature</span> <span class="operator">=</span> JsonPath.read(forecast, <span class="string">"$.properties.periods[0].temperature"</span>).toString();</span><br><span class="line"> <span class="type">String</span> <span class="variable">temperatureUnit</span> <span class="operator">=</span> JsonPath.read(forecast, <span class="string">"$.properties.periods[0].temperatureUnit"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">windSpeed</span> <span class="operator">=</span> JsonPath.read(forecast, <span class="string">"$.properties.periods[0].windSpeed"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">windDirection</span> <span class="operator">=</span> JsonPath.read(forecast, <span class="string">"$.properties.periods[0].windDirection"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 构建返回信息</span></span><br><span class="line"> <span class="keyword">return</span> String.format(<span class="string">"Temperature: %s°%s\nWind: %s %s\nForecast: %s"</span>,</span><br><span class="line"> temperature, temperatureUnit, windSpeed, windDirection, detailedForecast);</span><br><span class="line"> <span class="comment">// Returns detailed forecast including:</span></span><br><span class="line"> <span class="comment">// - Temperature and unit</span></span><br><span class="line"> <span class="comment">// - Wind speed and direction</span></span><br><span class="line"> <span class="comment">// - Detailed forecast description</span></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Tool(description = "Get weather alerts for a US state")</span></span><br><span class="line"> <span class="keyword">public</span> String <span class="title function_">getAlerts</span><span class="params">(<span class="meta">@ToolParam(description = "Two-letter US state code (e.g. CA, NY)")</span> String state)</span> {</span><br><span class="line"> </span><br><span class="line"> <span class="comment">// 获取指定州的天气警报</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">alertsResponse</span> <span class="operator">=</span> restClient.get()</span><br><span class="line"> .uri(<span class="string">"/alerts/active/area/{state}"</span>, state)</span><br><span class="line"> .retrieve()</span><br><span class="line"> .body(String.class);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 检查是否有警报</span></span><br><span class="line"> List<Map<String, Object>> features = JsonPath.read(alertsResponse, <span class="string">"$.features"</span>);</span><br><span class="line"> <span class="keyword">if</span> (features.isEmpty()) {</span><br><span class="line"> <span class="keyword">return</span> <span class="string">"当前没有活动警报。"</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 构建警报信息</span></span><br><span class="line"> <span class="type">StringBuilder</span> <span class="variable">alertInfo</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">StringBuilder</span>();</span><br><span class="line"> <span class="keyword">for</span> (Map<String, Object> feature : features) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">event</span> <span class="operator">=</span> JsonPath.read(feature, <span class="string">"$.properties.event"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">area</span> <span class="operator">=</span> JsonPath.read(feature, <span class="string">"$.properties.areaDesc"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">severity</span> <span class="operator">=</span> JsonPath.read(feature, <span class="string">"$.properties.severity"</span>); </span><br><span class="line"> <span class="type">String</span> <span class="variable">description</span> <span class="operator">=</span> JsonPath.read(feature, <span class="string">"$.properties.description"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">instruction</span> <span class="operator">=</span> JsonPath.read(feature, <span class="string">"$.properties.instruction"</span>);</span><br><span class="line"></span><br><span class="line"> alertInfo.append(<span class="string">"警报类型: "</span>).append(event).append(<span class="string">"\n"</span>);</span><br><span class="line"> alertInfo.append(<span class="string">"影响区域: "</span>).append(area).append(<span class="string">"\n"</span>);</span><br><span class="line"> alertInfo.append(<span class="string">"严重程度: "</span>).append(severity).append(<span class="string">"\n"</span>);</span><br><span class="line"> alertInfo.append(<span class="string">"描述: "</span>).append(description).append(<span class="string">"\n"</span>);</span><br><span class="line"> <span class="keyword">if</span> (instruction != <span class="literal">null</span>) {</span><br><span class="line"> alertInfo.append(<span class="string">"安全指示: "</span>).append(instruction).append(<span class="string">"\n"</span>);</span><br><span class="line"> }</span><br><span class="line"> alertInfo.append(<span class="string">"\n---\n\n"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> alertInfo.toString();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// ......</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>通过 <code>api.weather.gov</code> 来请求天气服务，给出结果<br>然后通过一个客户端来访问请求下</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> <span class="type">var</span> <span class="variable">stdioParams</span> <span class="operator">=</span> ServerParameters.builder(<span class="string">"java"</span>)</span><br><span class="line"> .args(<span class="string">"-jar"</span>, <span class="string">"/Users/shixuesen/Downloads/mcp-demo/target/mcp-demo-0.0.1-SNAPSHOT.jar"</span>)</span><br><span class="line"> .build();</span><br><span class="line"></span><br><span class="line"> <span class="type">var</span> <span class="variable">stdioTransport</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">StdioClientTransport</span>(stdioParams);</span><br><span class="line"></span><br><span class="line"> <span class="type">var</span> <span class="variable">mcpClient</span> <span class="operator">=</span> McpClient.sync(stdioTransport).build();</span><br><span class="line"></span><br><span class="line"> mcpClient.initialize();</span><br><span class="line"></span><br><span class="line"><span class="comment">// McpSchema.ListToolsResult toolsList = mcpClient.listTools();</span></span><br><span class="line"></span><br><span class="line"> McpSchema.<span class="type">CallToolResult</span> <span class="variable">weather</span> <span class="operator">=</span> mcpClient.callTool(</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">McpSchema</span>.CallToolRequest(<span class="string">"getWeatherForecastByLocation"</span>,</span><br><span class="line"> Map.of(<span class="string">"latitude"</span>, <span class="string">"47.6062"</span>, <span class="string">"longitude"</span>, <span class="string">"-122.3321"</span>)));</span><br><span class="line"> System.out.println(weather.content());</span><br><span class="line"></span><br><span class="line"> McpSchema.<span class="type">CallToolResult</span> <span class="variable">alert</span> <span class="operator">=</span> mcpClient.callTool(</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">McpSchema</span>.CallToolRequest(<span class="string">"getAlerts"</span>, Map.of(<span class="string">"state"</span>, <span class="string">"NY"</span>)));</span><br><span class="line"></span><br><span class="line"> mcpClient.closeGracefully();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这个就是实现了mcp的简单示例，几个问题，要注意java的多版本管理，我这边主要是用jdk1.8，切成 17需要对应改maven的配置等</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊一下 Java 的日志系列三</title>
|
|
<url>/2024/01/21/%E8%81%8A%E4%B8%80%E4%B8%8B-Java-%E7%9A%84%E6%97%A5%E5%BF%97%E7%B3%BB%E5%88%97%E4%B8%89/</url>
|
|
<content><![CDATA[<p>上周因为一些事情没有更新在这里，是因为新电脑还没到，手头没有把移动硬盘里的 time machine 恢复出来的机器，所以单独更了一篇在新建的一个 cloudflare page 服务上，总体体验还可以，就是有个小点后面可以讲一下，继续完善下 Java 的这个日志，或者说主要讲的是 logback<br>前面讲了logback 的初始化逻辑，后面就是我最开始来看这个的逻辑，日志的滚动策略的触发逻辑<br>我们常规的记录日志的方式比如</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">logger.info(<span class="string">"log some thing and biz info: {}"</span>,biz);</span><br></pre></td></tr></table></figure>
|
|
<p>这里就调用了<br><code>ch.qos.logback.classic.Logger#info(java.lang.String, java.lang.Object)</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">info</span><span class="params">(String format, Object arg)</span> {</span><br><span class="line"> filterAndLog_1(FQCN, <span class="literal">null</span>, Level.INFO, format, arg, <span class="literal">null</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里的 FQCN 就是这个类的全限定名</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">FQCN</span> <span class="operator">=</span> ch.qos.logback.classic.Logger.class.getName();</span><br></pre></td></tr></table></figure>
|
|
<p>这个具体的 filterAndLog_1 方法还是在同一个 Logger 类里的<br><code>ch.qos.logback.classic.Logger#filterAndLog_1</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">filterAndLog_1</span><span class="params">(<span class="keyword">final</span> String localFQCN, <span class="keyword">final</span> Marker marker, <span class="keyword">final</span> Level level, <span class="keyword">final</span> String msg, <span class="keyword">final</span> Object param, <span class="keyword">final</span> Throwable t)</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">final</span> <span class="type">FilterReply</span> <span class="variable">decision</span> <span class="operator">=</span> loggerContext.getTurboFilterChainDecision_1(marker, <span class="built_in">this</span>, level, msg, param, t);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (decision == FilterReply.NEUTRAL) {</span><br><span class="line"> <span class="keyword">if</span> (effectiveLevelInt > level.levelInt) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (decision == FilterReply.DENY) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> buildLoggingEventAndAppend(localFQCN, marker, level, msg, <span class="keyword">new</span> <span class="title class_">Object</span>[] { param }, t);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里面先是走过滤器逻辑，如果是拒绝的，那就不往下执行，也就不执行下面的写日志逻辑了，如果是中性的，那就判断日志级别是否符合，如果直接是拒绝就 return 了，如果是 ACCEPT 就往下执行了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">enum</span> <span class="title class_">FilterReply</span> {</span><br><span class="line"> DENY, NEUTRAL, ACCEPT;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后继续调用同样的类的方法<br><code>ch.qos.logback.classic.Logger#buildLoggingEventAndAppend</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">buildLoggingEventAndAppend</span><span class="params">(<span class="keyword">final</span> String localFQCN, <span class="keyword">final</span> Marker marker, <span class="keyword">final</span> Level level, <span class="keyword">final</span> String msg, <span class="keyword">final</span> Object[] params,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> Throwable t)</span> {</span><br><span class="line"> <span class="type">LoggingEvent</span> <span class="variable">le</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">LoggingEvent</span>(localFQCN, <span class="built_in">this</span>, level, msg, t, params);</span><br><span class="line"> le.setMarker(marker);</span><br><span class="line"> callAppenders(le);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>先把日志包装成 <code>LoggingEvent</code> ，然后调用<br><code>ch.qos.logback.classic.Logger#callAppenders</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">callAppenders</span><span class="params">(ILoggingEvent event)</span> {</span><br><span class="line"> <span class="type">int</span> <span class="variable">writes</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">Logger</span> <span class="variable">l</span> <span class="operator">=</span> <span class="built_in">this</span>; l != <span class="literal">null</span>; l = l.parent) {</span><br><span class="line"> writes += l.appendLoopOnAppenders(event);</span><br><span class="line"> <span class="keyword">if</span> (!l.additive) {</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// No appenders in hierarchy</span></span><br><span class="line"> <span class="keyword">if</span> (writes == <span class="number">0</span>) {</span><br><span class="line"> loggerContext.noAppenderDefinedWarning(<span class="built_in">this</span>);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里会按 Logger 调用 <code>appendLoopOnAppenders</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="type">int</span> <span class="title function_">appendLoopOnAppenders</span><span class="params">(ILoggingEvent event)</span> {</span><br><span class="line"> <span class="keyword">if</span> (aai != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> aai.appendLoopOnAppenders(event);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>继续调用事件的<br><code>ch.qos.logback.core.spi.AppenderAttachableImpl#appendLoopOnAppenders</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">int</span> <span class="title function_">appendLoopOnAppenders</span><span class="params">(E e)</span> {</span><br><span class="line"> <span class="type">int</span> <span class="variable">size</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">final</span> Appender<E>[] appenderArray = appenderList.asTypedArray();</span><br><span class="line"> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">len</span> <span class="operator">=</span> appenderArray.length;</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < len; i++) {</span><br><span class="line"> appenderArray[i].doAppend(e);</span><br><span class="line"> size++;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> size;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>因为 Logger 里有关联 appender，就具体调用关联的 appender 来添加日志<br>然后就接着调用<br><code>ch.qos.logback.core.UnsynchronizedAppenderBase#doAppend</code> 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">doAppend</span><span class="params">(E eventObject)</span> {</span><br><span class="line"> <span class="comment">// WARNING: The guard check MUST be the first statement in the</span></span><br><span class="line"> <span class="comment">// doAppend() method.</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">// prevent re-entry.</span></span><br><span class="line"> <span class="keyword">if</span> (Boolean.TRUE.equals(guard.get())) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> guard.set(Boolean.TRUE);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!<span class="built_in">this</span>.started) {</span><br><span class="line"> <span class="keyword">if</span> (statusRepeatCount++ < ALLOWED_REPEATS) {</span><br><span class="line"> addStatus(<span class="keyword">new</span> <span class="title class_">WarnStatus</span>(<span class="string">"Attempted to append to non started appender ["</span> + name + <span class="string">"]."</span>, <span class="built_in">this</span>));</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (getFilterChainDecision(eventObject) == FilterReply.DENY) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// ok, we now invoke derived class' implementation of append</span></span><br><span class="line"> <span class="built_in">this</span>.append(eventObject);</span><br><span class="line"></span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">if</span> (exceptionCount++ < ALLOWED_REPEATS) {</span><br><span class="line"> addError(<span class="string">"Appender ["</span> + name + <span class="string">"] failed to append."</span>, e);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> guard.set(Boolean.FALSE);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里先判断了 guard 的状态，guard 是个 ThreadLocal，是为了拦截异常的无限循环调用，这里有些疑惑的，看了一圈可能相关的是前面的 Logger 会往父级查找，然后如果不同层级的 Logger 关联了相同的 appender 的话，或者这个 doAppend 又被其他 appender 调用了，就可能会出现循环调用，然后会在开始实际的 append的之前先设置 guard 状态，<br>然后再执行 append，这里会走到<br><code>ch.qos.logback.core.OutputStreamAppender#append</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">append</span><span class="params">(E eventObject)</span> {</span><br><span class="line"> <span class="keyword">if</span> (!isStarted()) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> subAppend(eventObject);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>忘了补充下，我们这里使用的是<br><code>ch.qos.logback.core.rolling.RollingFileAppender</code><br>接下去就是调用的 <code>RollingFileAppender</code> 的 <code>subAppend</code> 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">subAppend</span><span class="params">(E event)</span> {</span><br><span class="line"> <span class="comment">// The roll-over check must precede actual writing. This is the</span></span><br><span class="line"> <span class="comment">// only correct behavior for time driven triggers.</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">// We need to synchronize on triggeringPolicy so that only one rollover</span></span><br><span class="line"> <span class="comment">// occurs at a time</span></span><br><span class="line"> <span class="keyword">synchronized</span> (triggeringPolicy) {</span><br><span class="line"> <span class="keyword">if</span> (triggeringPolicy.isTriggeringEvent(currentlyActiveFile, event)) {</span><br><span class="line"> rollover();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">super</span>.subAppend(event);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里就会先判断是否触发事件，<br>而这里就是主要的逻辑，看是否需要走 rollover，也就是具体的滚动逻辑</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">isTriggeringEvent</span><span class="params">(File activeFile, <span class="keyword">final</span> E event)</span> {</span><br><span class="line"></span><br><span class="line"> <span class="type">long</span> <span class="variable">time</span> <span class="operator">=</span> getCurrentTime();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// first check for roll-over based on time</span></span><br><span class="line"> <span class="keyword">if</span> (time >= nextCheck) {</span><br><span class="line"> <span class="type">Date</span> <span class="variable">dateInElapsedPeriod</span> <span class="operator">=</span> dateInCurrentPeriod;</span><br><span class="line"> elapsedPeriodsFileName = tbrp.fileNamePatternWithoutCompSuffix.convertMultipleArguments(dateInElapsedPeriod, currentPeriodsCounter);</span><br><span class="line"> currentPeriodsCounter = <span class="number">0</span>;</span><br><span class="line"> setDateInCurrentPeriod(time);</span><br><span class="line"> computeNextCheck();</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// next check for roll-over based on size</span></span><br><span class="line"> <span class="keyword">if</span> (invocationGate.isTooSoon(time)) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (activeFile == <span class="literal">null</span>) {</span><br><span class="line"> addWarn(<span class="string">"activeFile == null"</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (maxFileSize == <span class="literal">null</span>) {</span><br><span class="line"> addWarn(<span class="string">"maxFileSize = null"</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (activeFile.length() >= maxFileSize.getSize()) {</span><br><span class="line"></span><br><span class="line"> elapsedPeriodsFileName = tbrp.fileNamePatternWithoutCompSuffix.convertMultipleArguments(dateInCurrentPeriod, currentPeriodsCounter);</span><br><span class="line"> currentPeriodsCounter++;</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里主要就用到了 nextCheck 判断，如果通过判断就可以执行滚动了，然后就是后面的判断，是否太快了，也就是 isTooSoon 然后再看当前的活跃日志以及最大文件大小是否设置，然后如果当前的活跃日志的大小超过了配置也会触发滚动</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>给局域网中的Ubuntu固定下ip</title>
|
|
<url>/2024/06/02/%E7%BB%99%E5%B1%80%E5%9F%9F%E7%BD%91%E4%B8%AD%E7%9A%84Ubuntu%E5%9B%BA%E5%AE%9A%E4%B8%8Bip/</url>
|
|
<content><![CDATA[<p>将一个小点，在局域网中包括类似于homelab的或者仅仅只是搭一个用来当实验环境的，在局域网内机器比较少的时候大部分DHCP会给分配相同的ip，但是这不是一定的，当机器比较多，并且上下线比较频繁时就会出现ip地址变动，这对于一些使用场景比较讨厌，所以我想把ip固定下来，先简单引用下DHCP的原理，他分为下面几个阶段</p>
|
|
<h3 id="DHCP发现（DISCOVER）"><a href="#DHCP发现（DISCOVER）" class="headerlink" title="DHCP发现（DISCOVER）"></a>DHCP发现（DISCOVER）</h3><p>client在物理子网上发送广播来寻找可用的服务器。网络管理员可以配置一个本地路由来转发DHCP包给另一个子网上的DHCP服务器。该client实现生成一个目的地址为255.255.255.255或者一个子网广播地址的UDP包。</p>
|
|
<p>客户也可以申请它使用的最后一个IP地址（在下面的例子里为192.168.1.100）。如果该客户所在的网络中此IP仍然可用，服务器就可以准许该申请。否则，就要看该服务器是授权的还是非授权的。授权服务器会拒绝请求，使得客户立刻申请一个新的IP。非授权服务器仅仅忽略掉请求，导致一个客户端请求的超时，于是客户端就会放弃此请求而去申请一个新的IP地址。</p>
|
|
<h3 id="DHCP提供（OFFER）"><a href="#DHCP提供（OFFER）" class="headerlink" title="DHCP提供（OFFER）"></a>DHCP提供（OFFER）</h3><p>当DHCP服务器收到一个来自客户的IP租约请求时，它会提供一个IP租约。DHCP为客户保留一个IP地址，然后通过网络单播一个DHCPOFFER消息给客户。该消息包含客户的MAC地址、服务器提供的IP地址、子网掩码、租期以及提供IP的DHCP服务器的IP。</p>
|
|
<p>服务器基于在CHADDR字段指定的客户硬件地址来检查配置。这里的服务器，192.168.1.1，将IP地址指定于YIADDR字段。</p>
|
|
<h3 id="DHCP请求（REQUEST）"><a href="#DHCP请求（REQUEST）" class="headerlink" title="DHCP请求（REQUEST）"></a>DHCP请求（REQUEST）</h3><p>当客户PC收到一个IP租约提供时，它必须告诉所有其他的DHCP服务器它已经接受了一个租约提供。因此，该客户会发送一个DHCPREQUEST消息，其中包含提供租约的服务器的IP。当其他DHCP服务器收到了该消息后，它们会收回所有可能已提供给该客户的租约。然后它们把曾经给该客户保留的那个地址重新放回到可用地址池中，这样，它们就可以为其他计算机分配这个地址。任意数量的DHCP服务器都可以响应同一个IP租约请求，但是每一个客户网卡只能接受一个租约提供。</p>
|
|
<h3 id="DHCP确认（Acknowledge，ACK）"><a href="#DHCP确认（Acknowledge，ACK）" class="headerlink" title="DHCP确认（Acknowledge，ACK）"></a>DHCP确认（Acknowledge，ACK）</h3><p>当DHCP服务器收到来自客户的REQUEST消息后，它就开始了配置过程的最后阶段。这个响应阶段包括发送一个DHCPACK包给客户。这个包包含租期和客户可能请求的其他所有配置信息。这时候，TCP/IP配置过程就完成了。</p>
|
|
<p>该服务器响应请求并发送响应给客户。整个系统期望客户来根据选项来配置其网卡。</p>
|
|
<h3 id="DHCP释放（RELEASE）"><a href="#DHCP释放（RELEASE）" class="headerlink" title="DHCP释放（RELEASE）"></a>DHCP释放（RELEASE）</h3><p>客户端向DHCP服务器发送一个请求以释放DHCP资源，并注销其IP地址。鉴于客户端更多的时候并不清楚何时用户会将其从网络中移除，此协议不会托管“DHCP释放的发送”。</p>
|
|
<h3 id="DHCP-NAK"><a href="#DHCP-NAK" class="headerlink" title="DHCP NAK"></a>DHCP NAK</h3><p>服务器回复客户，客户要求的IP不能被分配。</p>
|
|
<h1 id="Ubuntu"><a href="#Ubuntu" class="headerlink" title="Ubuntu"></a>Ubuntu</h1><p>而对于Ubuntu，现在较新版本的设置方式也有点变化，比如我用的是22.04</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">cd /etc/netplan/</span><br></pre></td></tr></table></figure>
|
|
<p>先去这个目录下查看配置文件<br>我们要找的是这个 <code>00-installer-config.yaml</code> 文件<br><img data-src="https://img.nicksxs.me/blog/2vQ0nG.png"><br>原本的是长这样<br><img data-src="https://img.nicksxs.me/blog/pHOYwe.png"><br>默认是通过dhcp分配ip<br>我们要改成静态ip</p>
|
|
<figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="attr">network:</span></span><br><span class="line"> <span class="attr">renderer:</span> <span class="string">networkd</span></span><br><span class="line"> <span class="attr">ethernets:</span></span><br><span class="line"> <span class="attr">ens33:</span></span><br><span class="line"> <span class="attr">addresses:</span></span><br><span class="line"> <span class="bullet">-</span> <span class="number">192.168</span><span class="number">.1</span><span class="string">.xx/24</span></span><br><span class="line"> <span class="attr">nameservers:</span></span><br><span class="line"> <span class="attr">addresses:</span> [<span class="number">114.114</span><span class="number">.114</span><span class="number">.114</span>, <span class="number">233.5</span><span class="number">.5</span><span class="number">.5</span>]</span><br><span class="line"> <span class="attr">routes:</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">to:</span> <span class="string">default</span></span><br><span class="line"> <span class="attr">via:</span> <span class="number">192.168</span><span class="number">.1</span><span class="number">.1</span></span><br><span class="line"> <span class="attr">version:</span> <span class="number">2</span></span><br></pre></td></tr></table></figure>
|
|
<p>解释一下 ethernets中的ens33表示我们的网卡接口的标识<br>可以通过 <code>ip addr</code> 查看<br><img data-src="https://img.nicksxs.me/blog/1gipDk.png"><br>第一个是本地回环，第二个可以看下是否跟内网ip对应上，如果是的话就这个ens33<br>然后就是addresses填的地址，就是我们想要固定的ip地址，记得得是自己局域网网段的<br>上面只是个示例，然后是nameservers里的是dns服务器，也是按需填写<br>这里的114.114.114.114跟233.5.5.5分别是电信跟阿里云的，仅供参考<br>然后是routes就是网关地址，按自己的环境填写，网络没有任何特殊处理的一般就是路由器<br>接下去就是应用这个配置</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sudo netplan apply</span><br></pre></td></tr></table></figure>
|
|
<p>这样ip配置就生效了<br>可以通过<br><code>ip addr show ens33</code>查看</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>linux</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>linux</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊一下 RocketMQ 的 DefaultMQPushConsumer 源码</title>
|
|
<url>/2020/06/26/%E8%81%8A%E4%B8%80%E4%B8%8B-RocketMQ-%E7%9A%84-Consumer/</url>
|
|
<content><![CDATA[<p>首先看下官方的小 demo</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> <span class="keyword">throws</span> InterruptedException, MQClientException {</span><br><span class="line"></span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> * Instantiate with specified consumer group name.</span></span><br><span class="line"><span class="comment"> * 首先是new 一个对象出来，然后指定 Consumer 的 Group</span></span><br><span class="line"><span class="comment"> * 同一类Consumer的集合，这类Consumer通常消费同一类消息且消费逻辑一致。消费者组使得在消息消费方面，实现负载均衡和容错的目标变得非常容易。要注意的是，消费者组的消费者实例必须订阅完全相同的Topic。RocketMQ 支持两种消息模式：集群消费（Clustering）和广播消费（Broadcasting）。</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="type">DefaultMQPushConsumer</span> <span class="variable">consumer</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">DefaultMQPushConsumer</span>(<span class="string">"please_rename_unique_group_name_4"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> * Specify name server addresses.</span></span><br><span class="line"><span class="comment"> * <p/></span></span><br><span class="line"><span class="comment"> * 这里可以通知指定环境变量或者设置对象参数的形式指定名字空间服务的地址</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * Alternatively, you may specify name server addresses via exporting environmental variable: NAMESRV_ADDR</span></span><br><span class="line"><span class="comment"> * <pre></span></span><br><span class="line"><span class="comment"> * {@code</span></span><br><span class="line"><span class="comment"> * consumer.setNamesrvAddr("name-server1-ip:9876;name-server2-ip:9876");</span></span><br><span class="line"><span class="comment"> * }</span></span><br><span class="line"><span class="comment"> * </pre></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> * Specify where to start in case the specified consumer group is a brand new one.</span></span><br><span class="line"><span class="comment"> * 指定消费起始点</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_FIRST_OFFSET);</span><br><span class="line"></span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> * Subscribe one more more topics to consume.</span></span><br><span class="line"><span class="comment"> * 指定订阅的 topic 跟 tag，注意后面的是个表达式，可以以 tag1 || tag2 || tag3 传入</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> consumer.subscribe(<span class="string">"TopicTest"</span>, <span class="string">"*"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> * Register callback to execute on arrival of messages fetched from brokers.</span></span><br><span class="line"><span class="comment"> * 注册具体获得消息后的处理方法</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> consumer.registerMessageListener(<span class="keyword">new</span> <span class="title class_">MessageListenerConcurrently</span>() {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> ConsumeConcurrentlyStatus <span class="title function_">consumeMessage</span><span class="params">(List<MessageExt> msgs,</span></span><br><span class="line"><span class="params"> ConsumeConcurrentlyContext context)</span> {</span><br><span class="line"> System.out.printf(<span class="string">"%s Receive New Messages: %s %n"</span>, Thread.currentThread().getName(), msgs);</span><br><span class="line"> <span class="keyword">return</span> ConsumeConcurrentlyStatus.CONSUME_SUCCESS;</span><br><span class="line"> }</span><br><span class="line"> });</span><br><span class="line"></span><br><span class="line"> <span class="comment">/*</span></span><br><span class="line"><span class="comment"> * Launch the consumer instance.</span></span><br><span class="line"><span class="comment"> * 启动消费者</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> consumer.start();</span><br><span class="line"></span><br><span class="line"> System.out.printf(<span class="string">"Consumer Started.%n"</span>);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后就是看看 start 的过程了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * This method gets internal infrastructure readily to serve. Instances must call this method after configuration.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@throws</span> MQClientException if there is any client error.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">start</span><span class="params">()</span> <span class="keyword">throws</span> MQClientException {</span><br><span class="line"> setConsumerGroup(NamespaceUtil.wrapNamespace(<span class="built_in">this</span>.getNamespace(), <span class="built_in">this</span>.consumerGroup));</span><br><span class="line"> <span class="built_in">this</span>.defaultMQPushConsumerImpl.start();</span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> != traceDispatcher) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> traceDispatcher.start(<span class="built_in">this</span>.getNamesrvAddr(), <span class="built_in">this</span>.getAccessChannel());</span><br><span class="line"> } <span class="keyword">catch</span> (MQClientException e) {</span><br><span class="line"> log.warn(<span class="string">"trace dispatcher start failed "</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>具体的逻辑在<code>this.defaultMQPushConsumerImpl.start()</code>，这个 defaultMQPushConsumerImpl 就是</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Internal implementation. Most of the functions herein are delegated to it.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">final</span> <span class="keyword">transient</span> DefaultMQPushConsumerImpl defaultMQPushConsumerImpl;</span><br></pre></td></tr></table></figure>
|
|
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title function_">start</span><span class="params">()</span> <span class="keyword">throws</span> MQClientException {</span><br><span class="line"> <span class="keyword">switch</span> (<span class="built_in">this</span>.serviceState) {</span><br><span class="line"> <span class="keyword">case</span> CREATE_JUST:</span><br><span class="line"> log.info(<span class="string">"the consumer [{}] start beginning. messageModel={}, isUnitMode={}"</span>, <span class="built_in">this</span>.defaultMQPushConsumer.getConsumerGroup(),</span><br><span class="line"> <span class="built_in">this</span>.defaultMQPushConsumer.getMessageModel(), <span class="built_in">this</span>.defaultMQPushConsumer.isUnitMode());</span><br><span class="line"> <span class="comment">// 这里比较巧妙，相当于想设立了个屏障，防止并发启动，不过这里并不是悲观锁，也不算个严格的乐观锁</span></span><br><span class="line"> <span class="built_in">this</span>.serviceState = ServiceState.START_FAILED;</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.checkConfig();</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.copySubscription();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.defaultMQPushConsumer.getMessageModel() == MessageModel.CLUSTERING) {</span><br><span class="line"> <span class="built_in">this</span>.defaultMQPushConsumer.changeInstanceNameToPID();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 这个mQClientFactory，负责管理client（consumer、producer），并提供多中功能接口供各个Service（Rebalance、PullMessage等）调用；大部分逻辑均在这个类中完成</span></span><br><span class="line"> <span class="built_in">this</span>.mQClientFactory = MQClientManager.getInstance().getOrCreateMQClientInstance(<span class="built_in">this</span>.defaultMQPushConsumer, <span class="built_in">this</span>.rpcHook);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 这个 rebalanceImpl 主要负责决定，当前的consumer应该从哪些Queue中消费消息；</span></span><br><span class="line"> <span class="built_in">this</span>.rebalanceImpl.setConsumerGroup(<span class="built_in">this</span>.defaultMQPushConsumer.getConsumerGroup());</span><br><span class="line"> <span class="built_in">this</span>.rebalanceImpl.setMessageModel(<span class="built_in">this</span>.defaultMQPushConsumer.getMessageModel());</span><br><span class="line"> <span class="built_in">this</span>.rebalanceImpl.setAllocateMessageQueueStrategy(<span class="built_in">this</span>.defaultMQPushConsumer.getAllocateMessageQueueStrategy());</span><br><span class="line"> <span class="built_in">this</span>.rebalanceImpl.setmQClientFactory(<span class="built_in">this</span>.mQClientFactory);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 长连接，负责从broker处拉取消息，然后利用ConsumeMessageService回调用户的Listener执行消息消费逻辑</span></span><br><span class="line"> <span class="built_in">this</span>.pullAPIWrapper = <span class="keyword">new</span> <span class="title class_">PullAPIWrapper</span>(</span><br><span class="line"> mQClientFactory,</span><br><span class="line"> <span class="built_in">this</span>.defaultMQPushConsumer.getConsumerGroup(), isUnitMode());</span><br><span class="line"> <span class="built_in">this</span>.pullAPIWrapper.registerFilterMessageHook(filterMessageHookList);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.defaultMQPushConsumer.getOffsetStore() != <span class="literal">null</span>) {</span><br><span class="line"> <span class="built_in">this</span>.offsetStore = <span class="built_in">this</span>.defaultMQPushConsumer.getOffsetStore();</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">switch</span> (<span class="built_in">this</span>.defaultMQPushConsumer.getMessageModel()) {</span><br><span class="line"> <span class="keyword">case</span> BROADCASTING:</span><br><span class="line"> <span class="built_in">this</span>.offsetStore = <span class="keyword">new</span> <span class="title class_">LocalFileOffsetStore</span>(<span class="built_in">this</span>.mQClientFactory, <span class="built_in">this</span>.defaultMQPushConsumer.getConsumerGroup());</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">case</span> CLUSTERING:</span><br><span class="line"> <span class="built_in">this</span>.offsetStore = <span class="keyword">new</span> <span class="title class_">RemoteBrokerOffsetStore</span>(<span class="built_in">this</span>.mQClientFactory, <span class="built_in">this</span>.defaultMQPushConsumer.getConsumerGroup());</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">default</span>:</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">this</span>.defaultMQPushConsumer.setOffsetStore(<span class="built_in">this</span>.offsetStore);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// offsetStore 维护当前consumer的消费记录（offset）；有两种实现，Local和Rmote，Local存储在本地磁盘上，适用于BROADCASTING广播消费模式；而Remote则将消费进度存储在Broker上，适用于CLUSTERING集群消费模式；</span></span><br><span class="line"> <span class="built_in">this</span>.offsetStore.load();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.getMessageListenerInner() <span class="keyword">instanceof</span> MessageListenerOrderly) {</span><br><span class="line"> <span class="built_in">this</span>.consumeOrderly = <span class="literal">true</span>;</span><br><span class="line"> <span class="built_in">this</span>.consumeMessageService =</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">ConsumeMessageOrderlyService</span>(<span class="built_in">this</span>, (MessageListenerOrderly) <span class="built_in">this</span>.getMessageListenerInner());</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (<span class="built_in">this</span>.getMessageListenerInner() <span class="keyword">instanceof</span> MessageListenerConcurrently) {</span><br><span class="line"> <span class="built_in">this</span>.consumeOrderly = <span class="literal">false</span>;</span><br><span class="line"> <span class="built_in">this</span>.consumeMessageService =</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">ConsumeMessageConcurrentlyService</span>(<span class="built_in">this</span>, (MessageListenerConcurrently) <span class="built_in">this</span>.getMessageListenerInner());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 实现所谓的"Push-被动"消费机制；从Broker拉取的消息后，封装成ConsumeRequest提交给ConsumeMessageSerivce，此service负责回调用户的Listener消费消息；</span></span><br><span class="line"> <span class="built_in">this</span>.consumeMessageService.start();</span><br><span class="line"></span><br><span class="line"> <span class="type">boolean</span> <span class="variable">registerOK</span> <span class="operator">=</span> mQClientFactory.registerConsumer(<span class="built_in">this</span>.defaultMQPushConsumer.getConsumerGroup(), <span class="built_in">this</span>);</span><br><span class="line"> <span class="keyword">if</span> (!registerOK) {</span><br><span class="line"> <span class="built_in">this</span>.serviceState = ServiceState.CREATE_JUST;</span><br><span class="line"> <span class="built_in">this</span>.consumeMessageService.shutdown();</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">MQClientException</span>(<span class="string">"The consumer group["</span> + <span class="built_in">this</span>.defaultMQPushConsumer.getConsumerGroup()</span><br><span class="line"> + <span class="string">"] has been created before, specify another name please."</span> + FAQUrl.suggestTodo(FAQUrl.GROUP_NAME_DUPLICATE_URL),</span><br><span class="line"> <span class="literal">null</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> mQClientFactory.start();</span><br><span class="line"> log.info(<span class="string">"the consumer [{}] start OK."</span>, <span class="built_in">this</span>.defaultMQPushConsumer.getConsumerGroup());</span><br><span class="line"> <span class="built_in">this</span>.serviceState = ServiceState.RUNNING;</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">case</span> RUNNING:</span><br><span class="line"> <span class="keyword">case</span> START_FAILED:</span><br><span class="line"> <span class="keyword">case</span> SHUTDOWN_ALREADY:</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">MQClientException</span>(<span class="string">"The PushConsumer service state not OK, maybe started once, "</span></span><br><span class="line"> + <span class="built_in">this</span>.serviceState</span><br><span class="line"> + FAQUrl.suggestTodo(FAQUrl.CLIENT_SERVICE_NOT_OK),</span><br><span class="line"> <span class="literal">null</span>);</span><br><span class="line"> <span class="keyword">default</span>:</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.updateTopicSubscribeInfoWhenSubscriptionChanged();</span><br><span class="line"> <span class="built_in">this</span>.mQClientFactory.checkClientInBroker();</span><br><span class="line"> <span class="built_in">this</span>.mQClientFactory.sendHeartbeatToAllBrokerWithLock();</span><br><span class="line"> <span class="built_in">this</span>.mQClientFactory.rebalanceImmediately();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后我们往下看主要的目光聚焦<code>mQClientFactory.start()</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">start</span><span class="params">()</span> <span class="keyword">throws</span> MQClientException {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">synchronized</span> (<span class="built_in">this</span>) {</span><br><span class="line"> <span class="keyword">switch</span> (<span class="built_in">this</span>.serviceState) {</span><br><span class="line"> <span class="keyword">case</span> CREATE_JUST:</span><br><span class="line"> <span class="built_in">this</span>.serviceState = ServiceState.START_FAILED;</span><br><span class="line"> <span class="comment">// If not specified,looking address from name server</span></span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == <span class="built_in">this</span>.clientConfig.getNamesrvAddr()) {</span><br><span class="line"> <span class="built_in">this</span>.mQClientAPIImpl.fetchNameServerAddr();</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// Start request-response channel</span></span><br><span class="line"> <span class="comment">// 这里主要是初始化了个网络客户端</span></span><br><span class="line"> <span class="built_in">this</span>.mQClientAPIImpl.start();</span><br><span class="line"> <span class="comment">// Start various schedule tasks</span></span><br><span class="line"> <span class="comment">// 定时任务</span></span><br><span class="line"> <span class="built_in">this</span>.startScheduledTask();</span><br><span class="line"> <span class="comment">// Start pull service</span></span><br><span class="line"> <span class="comment">// 这里重点说下</span></span><br><span class="line"> <span class="built_in">this</span>.pullMessageService.start();</span><br><span class="line"> <span class="comment">// Start rebalance service</span></span><br><span class="line"> <span class="built_in">this</span>.rebalanceService.start();</span><br><span class="line"> <span class="comment">// Start push service</span></span><br><span class="line"> <span class="built_in">this</span>.defaultMQProducer.getDefaultMQProducerImpl().start(<span class="literal">false</span>);</span><br><span class="line"> log.info(<span class="string">"the client factory [{}] start OK"</span>, <span class="built_in">this</span>.clientId);</span><br><span class="line"> <span class="built_in">this</span>.serviceState = ServiceState.RUNNING;</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">case</span> START_FAILED:</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">MQClientException</span>(<span class="string">"The Factory object["</span> + <span class="built_in">this</span>.getClientId() + <span class="string">"] has been created before, and failed."</span>, <span class="literal">null</span>);</span><br><span class="line"> <span class="keyword">default</span>:</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>我们来看下这个 pullMessageService，org.apache.rocketmq.client.impl.consumer.PullMessageService，<br><img data-src="https://mystore-1255223546.cos.ap-chengdu.myqcloud.com/uPic/QdeiVv.png"><br>实现了 runnable 接口，<br>然后可以看到 run 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> log.info(<span class="built_in">this</span>.getServiceName() + <span class="string">" service started"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">while</span> (!<span class="built_in">this</span>.isStopped()) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">PullRequest</span> <span class="variable">pullRequest</span> <span class="operator">=</span> <span class="built_in">this</span>.pullRequestQueue.take();</span><br><span class="line"> <span class="built_in">this</span>.pullMessage(pullRequest);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException ignored) {</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> log.error(<span class="string">"Pull Message Service Run Method exception"</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> log.info(<span class="built_in">this</span>.getServiceName() + <span class="string">" service end"</span>);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>接着在看 pullMessage 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">pullMessage</span><span class="params">(<span class="keyword">final</span> PullRequest pullRequest)</span> {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">MQConsumerInner</span> <span class="variable">consumer</span> <span class="operator">=</span> <span class="built_in">this</span>.mQClientFactory.selectConsumer(pullRequest.getConsumerGroup());</span><br><span class="line"> <span class="keyword">if</span> (consumer != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">DefaultMQPushConsumerImpl</span> <span class="variable">impl</span> <span class="operator">=</span> (DefaultMQPushConsumerImpl) consumer;</span><br><span class="line"> impl.pullMessage(pullRequest);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> log.warn(<span class="string">"No matched consumer for the PullRequest {}, drop it"</span>, pullRequest);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>实际上调用了这个方法，这个方法很长，我在代码里注释下下每一段的功能</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">pullMessage</span><span class="params">(<span class="keyword">final</span> PullRequest pullRequest)</span> {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">ProcessQueue</span> <span class="variable">processQueue</span> <span class="operator">=</span> pullRequest.getProcessQueue();</span><br><span class="line"> <span class="comment">// 这里开始就是检查状态，确定是否往下执行</span></span><br><span class="line"> <span class="keyword">if</span> (processQueue.isDropped()) {</span><br><span class="line"> log.info(<span class="string">"the pull request[{}] is dropped."</span>, pullRequest.toString());</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> pullRequest.getProcessQueue().setLastPullTimestamp(System.currentTimeMillis());</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="built_in">this</span>.makeSureStateOK();</span><br><span class="line"> } <span class="keyword">catch</span> (MQClientException e) {</span><br><span class="line"> log.warn(<span class="string">"pullMessage exception, consumer state not ok"</span>, e);</span><br><span class="line"> <span class="built_in">this</span>.executePullRequestLater(pullRequest, pullTimeDelayMillsWhenException);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.isPause()) {</span><br><span class="line"> log.warn(<span class="string">"consumer was paused, execute pull request later. instanceName={}, group={}"</span>, <span class="built_in">this</span>.defaultMQPushConsumer.getInstanceName(), <span class="built_in">this</span>.defaultMQPushConsumer.getConsumerGroup());</span><br><span class="line"> <span class="built_in">this</span>.executePullRequestLater(pullRequest, PULL_TIME_DELAY_MILLS_WHEN_SUSPEND);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 这块其实是个类似于限流的功能块，对消息数量和消息大小做限制</span></span><br><span class="line"> <span class="type">long</span> <span class="variable">cachedMessageCount</span> <span class="operator">=</span> processQueue.getMsgCount().get();</span><br><span class="line"> <span class="type">long</span> <span class="variable">cachedMessageSizeInMiB</span> <span class="operator">=</span> processQueue.getMsgSize().get() / (<span class="number">1024</span> * <span class="number">1024</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (cachedMessageCount > <span class="built_in">this</span>.defaultMQPushConsumer.getPullThresholdForQueue()) {</span><br><span class="line"> <span class="built_in">this</span>.executePullRequestLater(pullRequest, PULL_TIME_DELAY_MILLS_WHEN_FLOW_CONTROL);</span><br><span class="line"> <span class="keyword">if</span> ((queueFlowControlTimes++ % <span class="number">1000</span>) == <span class="number">0</span>) {</span><br><span class="line"> log.warn(</span><br><span class="line"> <span class="string">"the cached message count exceeds the threshold {}, so do flow control, minOffset={}, maxOffset={}, count={}, size={} MiB, pullRequest={}, flowControlTimes={}"</span>,</span><br><span class="line"> <span class="built_in">this</span>.defaultMQPushConsumer.getPullThresholdForQueue(), processQueue.getMsgTreeMap().firstKey(), processQueue.getMsgTreeMap().lastKey(), cachedMessageCount, cachedMessageSizeInMiB, pullRequest, queueFlowControlTimes);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (cachedMessageSizeInMiB > <span class="built_in">this</span>.defaultMQPushConsumer.getPullThresholdSizeForQueue()) {</span><br><span class="line"> <span class="built_in">this</span>.executePullRequestLater(pullRequest, PULL_TIME_DELAY_MILLS_WHEN_FLOW_CONTROL);</span><br><span class="line"> <span class="keyword">if</span> ((queueFlowControlTimes++ % <span class="number">1000</span>) == <span class="number">0</span>) {</span><br><span class="line"> log.warn(</span><br><span class="line"> <span class="string">"the cached message size exceeds the threshold {} MiB, so do flow control, minOffset={}, maxOffset={}, count={}, size={} MiB, pullRequest={}, flowControlTimes={}"</span>,</span><br><span class="line"> <span class="built_in">this</span>.defaultMQPushConsumer.getPullThresholdSizeForQueue(), processQueue.getMsgTreeMap().firstKey(), processQueue.getMsgTreeMap().lastKey(), cachedMessageCount, cachedMessageSizeInMiB, pullRequest, queueFlowControlTimes);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 若不是顺序消费（即DefaultMQPushConsumerImpl.consumeOrderly等于false），则检查ProcessQueue对象的msgTreeMap:TreeMap<Long,MessageExt>变量的第一个key值与最后一个key值之间的差额，该key值表示查询的队列偏移量queueoffset；若差额大于阈值（由DefaultMQPushConsumer. consumeConcurrentlyMaxSpan指定，默认是2000），则调用PullMessageService.executePullRequestLater方法，在50毫秒之后重新将该PullRequest请求放入PullMessageService.pullRequestQueue队列中；并跳出该方法；这里的意思主要就是消息有堆积了，等会再来拉取</span></span><br><span class="line"> <span class="keyword">if</span> (!<span class="built_in">this</span>.consumeOrderly) {</span><br><span class="line"> <span class="keyword">if</span> (processQueue.getMaxSpan() > <span class="built_in">this</span>.defaultMQPushConsumer.getConsumeConcurrentlyMaxSpan()) {</span><br><span class="line"> <span class="built_in">this</span>.executePullRequestLater(pullRequest, PULL_TIME_DELAY_MILLS_WHEN_FLOW_CONTROL);</span><br><span class="line"> <span class="keyword">if</span> ((queueMaxSpanFlowControlTimes++ % <span class="number">1000</span>) == <span class="number">0</span>) {</span><br><span class="line"> log.warn(</span><br><span class="line"> <span class="string">"the queue's messages, span too long, so do flow control, minOffset={}, maxOffset={}, maxSpan={}, pullRequest={}, flowControlTimes={}"</span>,</span><br><span class="line"> processQueue.getMsgTreeMap().firstKey(), processQueue.getMsgTreeMap().lastKey(), processQueue.getMaxSpan(),</span><br><span class="line"> pullRequest, queueMaxSpanFlowControlTimes);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">if</span> (processQueue.isLocked()) {</span><br><span class="line"> <span class="keyword">if</span> (!pullRequest.isLockedFirst()) {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">long</span> <span class="variable">offset</span> <span class="operator">=</span> <span class="built_in">this</span>.rebalanceImpl.computePullFromWhere(pullRequest.getMessageQueue());</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">brokerBusy</span> <span class="operator">=</span> offset < pullRequest.getNextOffset();</span><br><span class="line"> log.info(<span class="string">"the first time to pull message, so fix offset from broker. pullRequest: {} NewOffset: {} brokerBusy: {}"</span>,</span><br><span class="line"> pullRequest, offset, brokerBusy);</span><br><span class="line"> <span class="keyword">if</span> (brokerBusy) {</span><br><span class="line"> log.info(<span class="string">"[NOTIFYME]the first time to pull message, but pull request offset larger than broker consume offset. pullRequest: {} NewOffset: {}"</span>,</span><br><span class="line"> pullRequest, offset);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> pullRequest.setLockedFirst(<span class="literal">true</span>);</span><br><span class="line"> pullRequest.setNextOffset(offset);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="built_in">this</span>.executePullRequestLater(pullRequest, pullTimeDelayMillsWhenException);</span><br><span class="line"> log.info(<span class="string">"pull message later because not locked in broker, {}"</span>, pullRequest);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 以PullRequest.messageQueue对象的topic值为参数从RebalanceImpl.subscriptionInner: ConcurrentHashMap, SubscriptionData>中获取对应的SubscriptionData对象，若该对象为null，考虑到并发的关系，调用executePullRequestLater方法，稍后重试；并跳出该方法；</span></span><br><span class="line"> <span class="keyword">final</span> <span class="type">SubscriptionData</span> <span class="variable">subscriptionData</span> <span class="operator">=</span> <span class="built_in">this</span>.rebalanceImpl.getSubscriptionInner().get(pullRequest.getMessageQueue().getTopic());</span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == subscriptionData) {</span><br><span class="line"> <span class="built_in">this</span>.executePullRequestLater(pullRequest, pullTimeDelayMillsWhenException);</span><br><span class="line"> log.warn(<span class="string">"find the consumer's subscription failed, {}"</span>, pullRequest);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">final</span> <span class="type">long</span> <span class="variable">beginTimestamp</span> <span class="operator">=</span> System.currentTimeMillis();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 异步拉取回调，先不讨论细节</span></span><br><span class="line"> <span class="type">PullCallback</span> <span class="variable">pullCallback</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">PullCallback</span>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">onSuccess</span><span class="params">(PullResult pullResult)</span> {</span><br><span class="line"> <span class="keyword">if</span> (pullResult != <span class="literal">null</span>) {</span><br><span class="line"> pullResult = DefaultMQPushConsumerImpl.<span class="built_in">this</span>.pullAPIWrapper.processPullResult(pullRequest.getMessageQueue(), pullResult,</span><br><span class="line"> subscriptionData);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">switch</span> (pullResult.getPullStatus()) {</span><br><span class="line"> <span class="keyword">case</span> FOUND:</span><br><span class="line"> <span class="type">long</span> <span class="variable">prevRequestOffset</span> <span class="operator">=</span> pullRequest.getNextOffset();</span><br><span class="line"> pullRequest.setNextOffset(pullResult.getNextBeginOffset());</span><br><span class="line"> <span class="type">long</span> <span class="variable">pullRT</span> <span class="operator">=</span> System.currentTimeMillis() - beginTimestamp;</span><br><span class="line"> DefaultMQPushConsumerImpl.<span class="built_in">this</span>.getConsumerStatsManager().incPullRT(pullRequest.getConsumerGroup(),</span><br><span class="line"> pullRequest.getMessageQueue().getTopic(), pullRT);</span><br><span class="line"></span><br><span class="line"> <span class="type">long</span> <span class="variable">firstMsgOffset</span> <span class="operator">=</span> Long.MAX_VALUE;</span><br><span class="line"> <span class="keyword">if</span> (pullResult.getMsgFoundList() == <span class="literal">null</span> || pullResult.getMsgFoundList().isEmpty()) {</span><br><span class="line"> DefaultMQPushConsumerImpl.<span class="built_in">this</span>.executePullRequestImmediately(pullRequest);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> firstMsgOffset = pullResult.getMsgFoundList().get(<span class="number">0</span>).getQueueOffset();</span><br><span class="line"></span><br><span class="line"> DefaultMQPushConsumerImpl.<span class="built_in">this</span>.getConsumerStatsManager().incPullTPS(pullRequest.getConsumerGroup(),</span><br><span class="line"> pullRequest.getMessageQueue().getTopic(), pullResult.getMsgFoundList().size());</span><br><span class="line"></span><br><span class="line"> <span class="type">boolean</span> <span class="variable">dispatchToConsume</span> <span class="operator">=</span> processQueue.putMessage(pullResult.getMsgFoundList());</span><br><span class="line"> DefaultMQPushConsumerImpl.<span class="built_in">this</span>.consumeMessageService.submitConsumeRequest(</span><br><span class="line"> pullResult.getMsgFoundList(),</span><br><span class="line"> processQueue,</span><br><span class="line"> pullRequest.getMessageQueue(),</span><br><span class="line"> dispatchToConsume);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (DefaultMQPushConsumerImpl.<span class="built_in">this</span>.defaultMQPushConsumer.getPullInterval() > <span class="number">0</span>) {</span><br><span class="line"> DefaultMQPushConsumerImpl.<span class="built_in">this</span>.executePullRequestLater(pullRequest,</span><br><span class="line"> DefaultMQPushConsumerImpl.<span class="built_in">this</span>.defaultMQPushConsumer.getPullInterval());</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> DefaultMQPushConsumerImpl.<span class="built_in">this</span>.executePullRequestImmediately(pullRequest);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (pullResult.getNextBeginOffset() < prevRequestOffset</span><br><span class="line"> || firstMsgOffset < prevRequestOffset) {</span><br><span class="line"> log.warn(</span><br><span class="line"> <span class="string">"[BUG] pull message result maybe data wrong, nextBeginOffset: {} firstMsgOffset: {} prevRequestOffset: {}"</span>,</span><br><span class="line"> pullResult.getNextBeginOffset(),</span><br><span class="line"> firstMsgOffset,</span><br><span class="line"> prevRequestOffset);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">case</span> NO_NEW_MSG:</span><br><span class="line"> pullRequest.setNextOffset(pullResult.getNextBeginOffset());</span><br><span class="line"></span><br><span class="line"> DefaultMQPushConsumerImpl.<span class="built_in">this</span>.correctTagsOffset(pullRequest);</span><br><span class="line"></span><br><span class="line"> DefaultMQPushConsumerImpl.<span class="built_in">this</span>.executePullRequestImmediately(pullRequest);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">case</span> NO_MATCHED_MSG:</span><br><span class="line"> pullRequest.setNextOffset(pullResult.getNextBeginOffset());</span><br><span class="line"></span><br><span class="line"> DefaultMQPushConsumerImpl.<span class="built_in">this</span>.correctTagsOffset(pullRequest);</span><br><span class="line"></span><br><span class="line"> DefaultMQPushConsumerImpl.<span class="built_in">this</span>.executePullRequestImmediately(pullRequest);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">case</span> OFFSET_ILLEGAL:</span><br><span class="line"> log.warn(<span class="string">"the pull request offset illegal, {} {}"</span>,</span><br><span class="line"> pullRequest.toString(), pullResult.toString());</span><br><span class="line"> pullRequest.setNextOffset(pullResult.getNextBeginOffset());</span><br><span class="line"></span><br><span class="line"> pullRequest.getProcessQueue().setDropped(<span class="literal">true</span>);</span><br><span class="line"> DefaultMQPushConsumerImpl.<span class="built_in">this</span>.executeTaskLater(<span class="keyword">new</span> <span class="title class_">Runnable</span>() {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> DefaultMQPushConsumerImpl.<span class="built_in">this</span>.offsetStore.updateOffset(pullRequest.getMessageQueue(),</span><br><span class="line"> pullRequest.getNextOffset(), <span class="literal">false</span>);</span><br><span class="line"></span><br><span class="line"> DefaultMQPushConsumerImpl.<span class="built_in">this</span>.offsetStore.persist(pullRequest.getMessageQueue());</span><br><span class="line"></span><br><span class="line"> DefaultMQPushConsumerImpl.<span class="built_in">this</span>.rebalanceImpl.removeProcessQueue(pullRequest.getMessageQueue());</span><br><span class="line"></span><br><span class="line"> log.warn(<span class="string">"fix the pull request offset, {}"</span>, pullRequest);</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> log.error(<span class="string">"executeTaskLater Exception"</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }, <span class="number">10000</span>);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">default</span>:</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">onException</span><span class="params">(Throwable e)</span> {</span><br><span class="line"> <span class="keyword">if</span> (!pullRequest.getMessageQueue().getTopic().startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX)) {</span><br><span class="line"> log.warn(<span class="string">"execute the pull request exception"</span>, e);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> DefaultMQPushConsumerImpl.<span class="built_in">this</span>.executePullRequestLater(pullRequest, pullTimeDelayMillsWhenException);</span><br><span class="line"> }</span><br><span class="line"> };</span><br><span class="line"> <span class="comment">// 如果为集群模式，即可置commitOffsetEnable为 true</span></span><br><span class="line"> <span class="type">boolean</span> <span class="variable">commitOffsetEnable</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="type">long</span> <span class="variable">commitOffsetValue</span> <span class="operator">=</span> <span class="number">0L</span>;</span><br><span class="line"> <span class="keyword">if</span> (MessageModel.CLUSTERING == <span class="built_in">this</span>.defaultMQPushConsumer.getMessageModel()) {</span><br><span class="line"> commitOffsetValue = <span class="built_in">this</span>.offsetStore.readOffset(pullRequest.getMessageQueue(), ReadOffsetType.READ_FROM_MEMORY);</span><br><span class="line"> <span class="keyword">if</span> (commitOffsetValue > <span class="number">0</span>) {</span><br><span class="line"> commitOffsetEnable = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 将上面获得的commitOffsetEnable更新到订阅关系里</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">subExpression</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">classFilter</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="type">SubscriptionData</span> <span class="variable">sd</span> <span class="operator">=</span> <span class="built_in">this</span>.rebalanceImpl.getSubscriptionInner().get(pullRequest.getMessageQueue().getTopic());</span><br><span class="line"> <span class="keyword">if</span> (sd != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.defaultMQPushConsumer.isPostSubscriptionWhenPull() && !sd.isClassFilterMode()) {</span><br><span class="line"> subExpression = sd.getSubString();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> classFilter = sd.isClassFilterMode();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 组成 sysFlag</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">sysFlag</span> <span class="operator">=</span> PullSysFlag.buildSysFlag(</span><br><span class="line"> commitOffsetEnable, <span class="comment">// commitOffset</span></span><br><span class="line"> <span class="literal">true</span>, <span class="comment">// suspend</span></span><br><span class="line"> subExpression != <span class="literal">null</span>, <span class="comment">// subscription</span></span><br><span class="line"> classFilter <span class="comment">// class filter</span></span><br><span class="line"> );</span><br><span class="line"> <span class="comment">// 调用真正的拉取消息接口</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="built_in">this</span>.pullAPIWrapper.pullKernelImpl(</span><br><span class="line"> pullRequest.getMessageQueue(),</span><br><span class="line"> subExpression,</span><br><span class="line"> subscriptionData.getExpressionType(),</span><br><span class="line"> subscriptionData.getSubVersion(),</span><br><span class="line"> pullRequest.getNextOffset(),</span><br><span class="line"> <span class="built_in">this</span>.defaultMQPushConsumer.getPullBatchSize(),</span><br><span class="line"> sysFlag,</span><br><span class="line"> commitOffsetValue,</span><br><span class="line"> BROKER_SUSPEND_MAX_TIME_MILLIS,</span><br><span class="line"> CONSUMER_TIMEOUT_MILLIS_WHEN_SUSPEND,</span><br><span class="line"> CommunicationMode.ASYNC,</span><br><span class="line"> pullCallback</span><br><span class="line"> );</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> log.error(<span class="string">"pullKernelImpl exception"</span>, e);</span><br><span class="line"> <span class="built_in">this</span>.executePullRequestLater(pullRequest, pullTimeDelayMillsWhenException);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>以下就是拉取消息的底层 api，不够不是特别复杂，主要是在找 broker，和设置请求参数</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> PullResult <span class="title function_">pullKernelImpl</span><span class="params">(</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> MessageQueue mq,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> String subExpression,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> String expressionType,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> <span class="type">long</span> subVersion,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> <span class="type">long</span> offset,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> <span class="type">int</span> maxNums,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> <span class="type">int</span> sysFlag,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> <span class="type">long</span> commitOffset,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> <span class="type">long</span> brokerSuspendMaxTimeMillis,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> <span class="type">long</span> timeoutMillis,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> CommunicationMode communicationMode,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> PullCallback pullCallback</span></span><br><span class="line"><span class="params">)</span> <span class="keyword">throws</span> MQClientException, RemotingException, MQBrokerException, InterruptedException {</span><br><span class="line"> <span class="type">FindBrokerResult</span> <span class="variable">findBrokerResult</span> <span class="operator">=</span></span><br><span class="line"> <span class="built_in">this</span>.mQClientFactory.findBrokerAddressInSubscribe(mq.getBrokerName(),</span><br><span class="line"> <span class="built_in">this</span>.recalculatePullFromWhichNode(mq), <span class="literal">false</span>);</span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == findBrokerResult) {</span><br><span class="line"> <span class="built_in">this</span>.mQClientFactory.updateTopicRouteInfoFromNameServer(mq.getTopic());</span><br><span class="line"> findBrokerResult =</span><br><span class="line"> <span class="built_in">this</span>.mQClientFactory.findBrokerAddressInSubscribe(mq.getBrokerName(),</span><br><span class="line"> <span class="built_in">this</span>.recalculatePullFromWhichNode(mq), <span class="literal">false</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (findBrokerResult != <span class="literal">null</span>) {</span><br><span class="line"> {</span><br><span class="line"> <span class="comment">// check version</span></span><br><span class="line"> <span class="keyword">if</span> (!ExpressionType.isTagType(expressionType)</span><br><span class="line"> && findBrokerResult.getBrokerVersion() < MQVersion.Version.V4_1_0_SNAPSHOT.ordinal()) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">MQClientException</span>(<span class="string">"The broker["</span> + mq.getBrokerName() + <span class="string">", "</span></span><br><span class="line"> + findBrokerResult.getBrokerVersion() + <span class="string">"] does not upgrade to support for filter message by "</span> + expressionType, <span class="literal">null</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="type">int</span> <span class="variable">sysFlagInner</span> <span class="operator">=</span> sysFlag;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (findBrokerResult.isSlave()) {</span><br><span class="line"> sysFlagInner = PullSysFlag.clearCommitOffsetFlag(sysFlagInner);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">PullMessageRequestHeader</span> <span class="variable">requestHeader</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">PullMessageRequestHeader</span>();</span><br><span class="line"> requestHeader.setConsumerGroup(<span class="built_in">this</span>.consumerGroup);</span><br><span class="line"> requestHeader.setTopic(mq.getTopic());</span><br><span class="line"> requestHeader.setQueueId(mq.getQueueId());</span><br><span class="line"> requestHeader.setQueueOffset(offset);</span><br><span class="line"> requestHeader.setMaxMsgNums(maxNums);</span><br><span class="line"> requestHeader.setSysFlag(sysFlagInner);</span><br><span class="line"> requestHeader.setCommitOffset(commitOffset);</span><br><span class="line"> requestHeader.setSuspendTimeoutMillis(brokerSuspendMaxTimeMillis);</span><br><span class="line"> requestHeader.setSubscription(subExpression);</span><br><span class="line"> requestHeader.setSubVersion(subVersion);</span><br><span class="line"> requestHeader.setExpressionType(expressionType);</span><br><span class="line"></span><br><span class="line"> <span class="type">String</span> <span class="variable">brokerAddr</span> <span class="operator">=</span> findBrokerResult.getBrokerAddr();</span><br><span class="line"> <span class="keyword">if</span> (PullSysFlag.hasClassFilterFlag(sysFlagInner)) {</span><br><span class="line"> brokerAddr = computPullFromWhichFilterServer(mq.getTopic(), brokerAddr);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">PullResult</span> <span class="variable">pullResult</span> <span class="operator">=</span> <span class="built_in">this</span>.mQClientFactory.getMQClientAPIImpl().pullMessage(</span><br><span class="line"> brokerAddr,</span><br><span class="line"> requestHeader,</span><br><span class="line"> timeoutMillis,</span><br><span class="line"> communicationMode,</span><br><span class="line"> pullCallback);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> pullResult;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">MQClientException</span>(<span class="string">"The broker["</span> + mq.getBrokerName() + <span class="string">"] not exist"</span>, <span class="literal">null</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>再看下一步的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> PullResult <span class="title function_">pullMessage</span><span class="params">(</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> String addr,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> PullMessageRequestHeader requestHeader,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> <span class="type">long</span> timeoutMillis,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> CommunicationMode communicationMode,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> PullCallback pullCallback</span></span><br><span class="line"><span class="params">)</span> <span class="keyword">throws</span> RemotingException, MQBrokerException, InterruptedException {</span><br><span class="line"> <span class="type">RemotingCommand</span> <span class="variable">request</span> <span class="operator">=</span> RemotingCommand.createRequestCommand(RequestCode.PULL_MESSAGE, requestHeader);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">switch</span> (communicationMode) {</span><br><span class="line"> <span class="keyword">case</span> ONEWAY:</span><br><span class="line"> <span class="keyword">assert</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">case</span> ASYNC:</span><br><span class="line"> <span class="built_in">this</span>.pullMessageAsync(addr, request, timeoutMillis, pullCallback);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">case</span> SYNC:</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.pullMessageSync(addr, request, timeoutMillis);</span><br><span class="line"> <span class="keyword">default</span>:</span><br><span class="line"> <span class="keyword">assert</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>通过 communicationMode 判断是同步拉取还是异步拉取，异步就调用</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">pullMessageAsync</span><span class="params">(</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> String addr,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> RemotingCommand request,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> <span class="type">long</span> timeoutMillis,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> PullCallback pullCallback</span></span><br><span class="line"><span class="params"> )</span> <span class="keyword">throws</span> RemotingException, InterruptedException {</span><br><span class="line"> <span class="built_in">this</span>.remotingClient.invokeAsync(addr, request, timeoutMillis, <span class="keyword">new</span> <span class="title class_">InvokeCallback</span>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">operationComplete</span><span class="params">(ResponseFuture responseFuture)</span> {</span><br><span class="line"> 异步</span><br><span class="line"> <span class="type">RemotingCommand</span> <span class="variable">response</span> <span class="operator">=</span> responseFuture.getResponseCommand();</span><br><span class="line"> <span class="keyword">if</span> (response != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">PullResult</span> <span class="variable">pullResult</span> <span class="operator">=</span> MQClientAPIImpl.<span class="built_in">this</span>.processPullResponse(response);</span><br><span class="line"> <span class="keyword">assert</span> pullResult != <span class="literal">null</span>;</span><br><span class="line"> pullCallback.onSuccess(pullResult);</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> pullCallback.onException(e);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">if</span> (!responseFuture.isSendRequestOK()) {</span><br><span class="line"> pullCallback.onException(<span class="keyword">new</span> <span class="title class_">MQClientException</span>(<span class="string">"send request failed to "</span> + addr + <span class="string">". Request: "</span> + request, responseFuture.getCause()));</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (responseFuture.isTimeout()) {</span><br><span class="line"> pullCallback.onException(<span class="keyword">new</span> <span class="title class_">MQClientException</span>(<span class="string">"wait response from "</span> + addr + <span class="string">" timeout :"</span> + responseFuture.getTimeoutMillis() + <span class="string">"ms"</span> + <span class="string">". Request: "</span> + request,</span><br><span class="line"> responseFuture.getCause()));</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> pullCallback.onException(<span class="keyword">new</span> <span class="title class_">MQClientException</span>(<span class="string">"unknown reason. addr: "</span> + addr + <span class="string">", timeoutMillis: "</span> + timeoutMillis + <span class="string">". Request: "</span> + request, responseFuture.getCause()));</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> });</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>并且会调用前面 pullCallback 的onSuccess和onException方法，同步的就是调用</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> PullResult <span class="title function_">pullMessageSync</span><span class="params">(</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> String addr,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> RemotingCommand request,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> <span class="type">long</span> timeoutMillis</span></span><br><span class="line"><span class="params"> )</span> <span class="keyword">throws</span> RemotingException, InterruptedException, MQBrokerException {</span><br><span class="line"> <span class="type">RemotingCommand</span> <span class="variable">response</span> <span class="operator">=</span> <span class="built_in">this</span>.remotingClient.invokeSync(addr, request, timeoutMillis);</span><br><span class="line"> <span class="keyword">assert</span> response != <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.processPullResponse(response);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后就是这个 remotingClient 的 invokeAsync 跟 invokeSync 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">invokeAsync</span><span class="params">(String addr, RemotingCommand request, <span class="type">long</span> timeoutMillis, InvokeCallback invokeCallback)</span></span><br><span class="line"> <span class="keyword">throws</span> InterruptedException, RemotingConnectException, RemotingTooMuchRequestException, RemotingTimeoutException,</span><br><span class="line"> RemotingSendRequestException {</span><br><span class="line"> <span class="type">long</span> <span class="variable">beginStartTime</span> <span class="operator">=</span> System.currentTimeMillis();</span><br><span class="line"> <span class="keyword">final</span> <span class="type">Channel</span> <span class="variable">channel</span> <span class="operator">=</span> <span class="built_in">this</span>.getAndCreateChannel(addr);</span><br><span class="line"> <span class="keyword">if</span> (channel != <span class="literal">null</span> && channel.isActive()) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> doBeforeRpcHooks(addr, request);</span><br><span class="line"> <span class="type">long</span> <span class="variable">costTime</span> <span class="operator">=</span> System.currentTimeMillis() - beginStartTime;</span><br><span class="line"> <span class="keyword">if</span> (timeoutMillis < costTime) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">RemotingTooMuchRequestException</span>(<span class="string">"invokeAsync call timeout"</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">this</span>.invokeAsyncImpl(channel, request, timeoutMillis - costTime, invokeCallback);</span><br><span class="line"> } <span class="keyword">catch</span> (RemotingSendRequestException e) {</span><br><span class="line"> log.warn(<span class="string">"invokeAsync: send request exception, so close the channel[{}]"</span>, addr);</span><br><span class="line"> <span class="built_in">this</span>.closeChannel(addr, channel);</span><br><span class="line"> <span class="keyword">throw</span> e;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="built_in">this</span>.closeChannel(addr, channel);</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">RemotingConnectException</span>(addr);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> RemotingCommand <span class="title function_">invokeSync</span><span class="params">(String addr, <span class="keyword">final</span> RemotingCommand request, <span class="type">long</span> timeoutMillis)</span></span><br><span class="line"> <span class="keyword">throws</span> InterruptedException, RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException {</span><br><span class="line"> <span class="type">long</span> <span class="variable">beginStartTime</span> <span class="operator">=</span> System.currentTimeMillis();</span><br><span class="line"> <span class="keyword">final</span> <span class="type">Channel</span> <span class="variable">channel</span> <span class="operator">=</span> <span class="built_in">this</span>.getAndCreateChannel(addr);</span><br><span class="line"> <span class="keyword">if</span> (channel != <span class="literal">null</span> && channel.isActive()) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> doBeforeRpcHooks(addr, request);</span><br><span class="line"> <span class="type">long</span> <span class="variable">costTime</span> <span class="operator">=</span> System.currentTimeMillis() - beginStartTime;</span><br><span class="line"> <span class="keyword">if</span> (timeoutMillis < costTime) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">RemotingTimeoutException</span>(<span class="string">"invokeSync call timeout"</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="type">RemotingCommand</span> <span class="variable">response</span> <span class="operator">=</span> <span class="built_in">this</span>.invokeSyncImpl(channel, request, timeoutMillis - costTime);</span><br><span class="line"> doAfterRpcHooks(RemotingHelper.parseChannelRemoteAddr(channel), request, response);</span><br><span class="line"> <span class="keyword">return</span> response;</span><br><span class="line"> } <span class="keyword">catch</span> (RemotingSendRequestException e) {</span><br><span class="line"> log.warn(<span class="string">"invokeSync: send request exception, so close the channel[{}]"</span>, addr);</span><br><span class="line"> <span class="built_in">this</span>.closeChannel(addr, channel);</span><br><span class="line"> <span class="keyword">throw</span> e;</span><br><span class="line"> } <span class="keyword">catch</span> (RemotingTimeoutException e) {</span><br><span class="line"> <span class="keyword">if</span> (nettyClientConfig.isClientCloseSocketIfTimeout()) {</span><br><span class="line"> <span class="built_in">this</span>.closeChannel(addr, channel);</span><br><span class="line"> log.warn(<span class="string">"invokeSync: close socket because of timeout, {}ms, {}"</span>, timeoutMillis, addr);</span><br><span class="line"> }</span><br><span class="line"> log.warn(<span class="string">"invokeSync: wait response timeout exception, the channel[{}]"</span>, addr);</span><br><span class="line"> <span class="keyword">throw</span> e;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="built_in">this</span>.closeChannel(addr, channel);</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">RemotingConnectException</span>(addr);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>再往下看</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> RemotingCommand <span class="title function_">invokeSyncImpl</span><span class="params">(<span class="keyword">final</span> Channel channel, <span class="keyword">final</span> RemotingCommand request,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> <span class="type">long</span> timeoutMillis)</span></span><br><span class="line"> <span class="keyword">throws</span> InterruptedException, RemotingSendRequestException, RemotingTimeoutException {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">opaque</span> <span class="operator">=</span> request.getOpaque();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> 同步跟异步都是会把结果用ResponseFuture抱起来</span><br><span class="line"> <span class="keyword">final</span> <span class="type">ResponseFuture</span> <span class="variable">responseFuture</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ResponseFuture</span>(channel, opaque, timeoutMillis, <span class="literal">null</span>, <span class="literal">null</span>);</span><br><span class="line"> <span class="built_in">this</span>.responseTable.put(opaque, responseFuture);</span><br><span class="line"> <span class="keyword">final</span> <span class="type">SocketAddress</span> <span class="variable">addr</span> <span class="operator">=</span> channel.remoteAddress();</span><br><span class="line"> channel.writeAndFlush(request).addListener(<span class="keyword">new</span> <span class="title class_">ChannelFutureListener</span>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">operationComplete</span><span class="params">(ChannelFuture f)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="keyword">if</span> (f.isSuccess()) {</span><br><span class="line"> responseFuture.setSendRequestOK(<span class="literal">true</span>);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> responseFuture.setSendRequestOK(<span class="literal">false</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> responseTable.remove(opaque);</span><br><span class="line"> responseFuture.setCause(f.cause());</span><br><span class="line"> responseFuture.putResponse(<span class="literal">null</span>);</span><br><span class="line"> log.warn(<span class="string">"send a request command to channel <"</span> + addr + <span class="string">"> failed."</span>);</span><br><span class="line"> }</span><br><span class="line"> });</span><br><span class="line"> <span class="comment">// 区别是同步的是在这等待</span></span><br><span class="line"> <span class="type">RemotingCommand</span> <span class="variable">responseCommand</span> <span class="operator">=</span> responseFuture.waitResponse(timeoutMillis);</span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == responseCommand) {</span><br><span class="line"> <span class="keyword">if</span> (responseFuture.isSendRequestOK()) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">RemotingTimeoutException</span>(RemotingHelper.parseSocketAddressAddr(addr), timeoutMillis,</span><br><span class="line"> responseFuture.getCause());</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">RemotingSendRequestException</span>(RemotingHelper.parseSocketAddressAddr(addr), responseFuture.getCause());</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> responseCommand;</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="built_in">this</span>.responseTable.remove(opaque);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">invokeAsyncImpl</span><span class="params">(<span class="keyword">final</span> Channel channel, <span class="keyword">final</span> RemotingCommand request, <span class="keyword">final</span> <span class="type">long</span> timeoutMillis,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> InvokeCallback invokeCallback)</span></span><br><span class="line"> <span class="keyword">throws</span> InterruptedException, RemotingTooMuchRequestException, RemotingTimeoutException, RemotingSendRequestException {</span><br><span class="line"> <span class="type">long</span> <span class="variable">beginStartTime</span> <span class="operator">=</span> System.currentTimeMillis();</span><br><span class="line"> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">opaque</span> <span class="operator">=</span> request.getOpaque();</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">acquired</span> <span class="operator">=</span> <span class="built_in">this</span>.semaphoreAsync.tryAcquire(timeoutMillis, TimeUnit.MILLISECONDS);</span><br><span class="line"> <span class="keyword">if</span> (acquired) {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">SemaphoreReleaseOnlyOnce</span> <span class="variable">once</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SemaphoreReleaseOnlyOnce</span>(<span class="built_in">this</span>.semaphoreAsync);</span><br><span class="line"> <span class="type">long</span> <span class="variable">costTime</span> <span class="operator">=</span> System.currentTimeMillis() - beginStartTime;</span><br><span class="line"> <span class="keyword">if</span> (timeoutMillis < costTime) {</span><br><span class="line"> once.release();</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">RemotingTimeoutException</span>(<span class="string">"invokeAsyncImpl call timeout"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">final</span> <span class="type">ResponseFuture</span> <span class="variable">responseFuture</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ResponseFuture</span>(channel, opaque, timeoutMillis - costTime, invokeCallback, once);</span><br><span class="line"> <span class="built_in">this</span>.responseTable.put(opaque, responseFuture);</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> channel.writeAndFlush(request).addListener(<span class="keyword">new</span> <span class="title class_">ChannelFutureListener</span>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">operationComplete</span><span class="params">(ChannelFuture f)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="keyword">if</span> (f.isSuccess()) {</span><br><span class="line"> responseFuture.setSendRequestOK(<span class="literal">true</span>);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> requestFail(opaque);</span><br><span class="line"> log.warn(<span class="string">"send a request command to channel <{}> failed."</span>, RemotingHelper.parseChannelRemoteAddr(channel));</span><br><span class="line"> }</span><br><span class="line"> });</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> responseFuture.release();</span><br><span class="line"> log.warn(<span class="string">"send a request command to channel <"</span> + RemotingHelper.parseChannelRemoteAddr(channel) + <span class="string">"> Exception"</span>, e);</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">RemotingSendRequestException</span>(RemotingHelper.parseChannelRemoteAddr(channel), e);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">if</span> (timeoutMillis <= <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">RemotingTooMuchRequestException</span>(<span class="string">"invokeAsyncImpl invoke too fast"</span>);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">info</span> <span class="operator">=</span></span><br><span class="line"> String.format(<span class="string">"invokeAsyncImpl tryAcquire semaphore timeout, %dms, waiting thread nums: %d semaphoreAsyncValue: %d"</span>,</span><br><span class="line"> timeoutMillis,</span><br><span class="line"> <span class="built_in">this</span>.semaphoreAsync.getQueueLength(),</span><br><span class="line"> <span class="built_in">this</span>.semaphoreAsync.availablePermits()</span><br><span class="line"> );</span><br><span class="line"> log.warn(info);</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">RemotingTimeoutException</span>(info);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
|
|
]]></content>
|
|
<categories>
|
|
<category>MQ</category>
|
|
<category>RocketMQ</category>
|
|
<category>消息队列</category>
|
|
<category>RocketMQ</category>
|
|
<category>中间件</category>
|
|
<category>RocketMQ</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>MQ</tag>
|
|
<tag>消息队列</tag>
|
|
<tag>RocketMQ</tag>
|
|
<tag>削峰填谷</tag>
|
|
<tag>中间件</tag>
|
|
<tag>DefaultMQPushConsumer</tag>
|
|
<tag>源码解析</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊一下 Java 的日志系列二</title>
|
|
<url>/2024/01/14/%E8%81%8A%E4%B8%80%E4%B8%8B-Java-%E7%9A%84%E6%97%A5%E5%BF%97%E7%B3%BB%E5%88%97%E4%BA%8C/</url>
|
|
<content><![CDATA[<p>log 初始化过程，首先是在启动类里会获取 logger</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">Log</span> <span class="variable">logger</span> <span class="operator">=</span> LogFactory.getLog(SpringApplication.class);</span><br></pre></td></tr></table></figure>
|
|
<p>然后是</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> Log <span class="title function_">getLog</span><span class="params">(Class clazz)</span> <span class="keyword">throws</span> LogConfigurationException {</span><br><span class="line"> <span class="keyword">return</span> getFactory().getInstance(clazz);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>继续下去是<br><code>org.apache.commons.logging.impl.SLF4JLogFactory#getInstance(java.lang.Class)</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Log <span class="title function_">getInstance</span><span class="params">(Class clazz)</span> <span class="keyword">throws</span> LogConfigurationException {</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.getInstance(clazz.getName());</span><br><span class="line">}</span><br><span class="line"><span class="keyword">public</span> Log <span class="title function_">getInstance</span><span class="params">(String name)</span> <span class="keyword">throws</span> LogConfigurationException {</span><br><span class="line"> <span class="type">Log</span> <span class="variable">instance</span> <span class="operator">=</span> (Log)<span class="built_in">this</span>.loggerMap.get(name);</span><br><span class="line"> <span class="keyword">if</span> (instance != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> instance;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="type">Logger</span> <span class="variable">slf4jLogger</span> <span class="operator">=</span> LoggerFactory.getLogger(name);</span><br><span class="line"> Object newInstance;</span><br><span class="line"> <span class="keyword">if</span> (slf4jLogger <span class="keyword">instanceof</span> LocationAwareLogger) {</span><br><span class="line"> newInstance = <span class="keyword">new</span> <span class="title class_">SLF4JLocationAwareLog</span>((LocationAwareLogger)slf4jLogger);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> newInstance = <span class="keyword">new</span> <span class="title class_">SLF4JLog</span>(slf4jLogger);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">Log</span> <span class="variable">oldInstance</span> <span class="operator">=</span> (Log)<span class="built_in">this</span>.loggerMap.putIfAbsent(name, newInstance);</span><br><span class="line"> <span class="keyword">return</span> (Log)(oldInstance == <span class="literal">null</span> ? newInstance : oldInstance);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后再往下处理就是 <code>LoggerFactory.getLogger(name);</code> 了，跟普通的获取 logger 一样</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> Logger <span class="title function_">getLogger</span><span class="params">(String name)</span> {</span><br><span class="line"> <span class="type">ILoggerFactory</span> <span class="variable">iLoggerFactory</span> <span class="operator">=</span> getILoggerFactory();</span><br><span class="line"> <span class="keyword">return</span> iLoggerFactory.getLogger(name);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>继续下去会先判断是否已初始化</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> ILoggerFactory <span class="title function_">getILoggerFactory</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (INITIALIZATION_STATE == UNINITIALIZED) {</span><br><span class="line"> <span class="keyword">synchronized</span> (LoggerFactory.class) {</span><br><span class="line"> <span class="keyword">if</span> (INITIALIZATION_STATE == UNINITIALIZED) {</span><br><span class="line"> INITIALIZATION_STATE = ONGOING_INITIALIZATION;</span><br><span class="line"> performInitialization();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">switch</span> (INITIALIZATION_STATE) {</span><br><span class="line"> <span class="keyword">case</span> SUCCESSFUL_INITIALIZATION:</span><br><span class="line"> <span class="keyword">return</span> StaticLoggerBinder.getSingleton().getLoggerFactory();</span><br><span class="line"> <span class="keyword">case</span> NOP_FALLBACK_INITIALIZATION:</span><br><span class="line"> <span class="keyword">return</span> NOP_FALLBACK_FACTORY;</span><br><span class="line"> <span class="keyword">case</span> FAILED_INITIALIZATION:</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalStateException</span>(UNSUCCESSFUL_INIT_MSG);</span><br><span class="line"> <span class="keyword">case</span> ONGOING_INITIALIZATION:</span><br><span class="line"> <span class="comment">// support re-entrant behavior.</span></span><br><span class="line"> <span class="comment">// See also http://jira.qos.ch/browse/SLF4J-97</span></span><br><span class="line"> <span class="keyword">return</span> SUBST_FACTORY;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalStateException</span>(<span class="string">"Unreachable code"</span>);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>第一次调用的话就会走到 <code>org.slf4j.LoggerFactory#performInitialization</code> 里面是先调用绑定，这也是 slf4j 门面模式的特点</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">final</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">performInitialization</span><span class="params">()</span> {</span><br><span class="line"> bind();</span><br><span class="line"> <span class="keyword">if</span> (INITIALIZATION_STATE == SUCCESSFUL_INITIALIZATION) {</span><br><span class="line"> versionSanityCheck();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">final</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">bind</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> Set<URL> staticLoggerBinderPathSet = <span class="literal">null</span>;</span><br><span class="line"> <span class="comment">// skip check under android, see also</span></span><br><span class="line"> <span class="comment">// http://jira.qos.ch/browse/SLF4J-328</span></span><br><span class="line"> <span class="keyword">if</span> (!isAndroid()) {</span><br><span class="line"> staticLoggerBinderPathSet = findPossibleStaticLoggerBinderPathSet();</span><br><span class="line"> reportMultipleBindingAmbiguity(staticLoggerBinderPathSet);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// the next line does the binding</span></span><br><span class="line"> StaticLoggerBinder.getSingleton();</span><br><span class="line"> INITIALIZATION_STATE = SUCCESSFUL_INITIALIZATION;</span><br><span class="line"> reportActualBinding(staticLoggerBinderPathSet);</span><br><span class="line"> fixSubstituteLoggers();</span><br><span class="line"> replayEvents();</span><br><span class="line"> <span class="comment">// release all resources in SUBST_FACTORY</span></span><br><span class="line"> SUBST_FACTORY.clear();</span><br><span class="line"> } <span class="keyword">catch</span> (NoClassDefFoundError ncde) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">msg</span> <span class="operator">=</span> ncde.getMessage();</span><br><span class="line"> <span class="keyword">if</span> (messageContainsOrgSlf4jImplStaticLoggerBinder(msg)) {</span><br><span class="line"> INITIALIZATION_STATE = NOP_FALLBACK_INITIALIZATION;</span><br><span class="line"> Util.report(<span class="string">"Failed to load class \"org.slf4j.impl.StaticLoggerBinder\"."</span>);</span><br><span class="line"> Util.report(<span class="string">"Defaulting to no-operation (NOP) logger implementation"</span>);</span><br><span class="line"> Util.report(<span class="string">"See "</span> + NO_STATICLOGGERBINDER_URL + <span class="string">" for further details."</span>);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> failedBinding(ncde);</span><br><span class="line"> <span class="keyword">throw</span> ncde;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (java.lang.NoSuchMethodError nsme) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">msg</span> <span class="operator">=</span> nsme.getMessage();</span><br><span class="line"> <span class="keyword">if</span> (msg != <span class="literal">null</span> && msg.contains(<span class="string">"org.slf4j.impl.StaticLoggerBinder.getSingleton()"</span>)) {</span><br><span class="line"> INITIALIZATION_STATE = FAILED_INITIALIZATION;</span><br><span class="line"> Util.report(<span class="string">"slf4j-api 1.6.x (or later) is incompatible with this binding."</span>);</span><br><span class="line"> Util.report(<span class="string">"Your binding is version 1.5.5 or earlier."</span>);</span><br><span class="line"> Util.report(<span class="string">"Upgrade your binding to version 1.6.x."</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">throw</span> nsme;</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> failedBinding(e);</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalStateException</span>(<span class="string">"Unexpected initialization failure"</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>非安卓环境会进入第一个 if 逻辑，先要去找static_logger_binder</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">static</span> Set<URL> <span class="title function_">findPossibleStaticLoggerBinderPathSet</span><span class="params">()</span> {</span><br><span class="line"> <span class="comment">// use Set instead of list in order to deal with bug #138</span></span><br><span class="line"> <span class="comment">// LinkedHashSet appropriate here because it preserves insertion order</span></span><br><span class="line"> <span class="comment">// during iteration</span></span><br><span class="line"> Set<URL> staticLoggerBinderPathSet = <span class="keyword">new</span> <span class="title class_">LinkedHashSet</span><URL>();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">ClassLoader</span> <span class="variable">loggerFactoryClassLoader</span> <span class="operator">=</span> LoggerFactory.class.getClassLoader();</span><br><span class="line"> Enumeration<URL> paths;</span><br><span class="line"> <span class="keyword">if</span> (loggerFactoryClassLoader == <span class="literal">null</span>) {</span><br><span class="line"> paths = ClassLoader.getSystemResources(STATIC_LOGGER_BINDER_PATH);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> paths = loggerFactoryClassLoader.getResources(STATIC_LOGGER_BINDER_PATH);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">while</span> (paths.hasMoreElements()) {</span><br><span class="line"> <span class="type">URL</span> <span class="variable">path</span> <span class="operator">=</span> paths.nextElement();</span><br><span class="line"> staticLoggerBinderPathSet.add(path);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (IOException ioe) {</span><br><span class="line"> Util.report(<span class="string">"Error getting resources from path"</span>, ioe);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> staticLoggerBinderPathSet;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>上面的 <code>STATIC_LOGGER_BINDER_PATH</code><br>就是 <code>org/slf4j/impl/StaticLoggerBinder.class</code> 如果找不到在外层方法就会抛出 <code>NoSuchMethodError</code> 错误，然后就是通过 <code>StaticLoggerBinder.getSingleton();</code> 获取 <code>StaticLoggerBinder</code> 实例了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> StaticLoggerBinder <span class="title function_">getSingleton</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> SINGLETON;</span><br><span class="line">}</span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="type">StaticLoggerBinder</span> <span class="variable">SINGLETON</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">StaticLoggerBinder</span>();</span><br></pre></td></tr></table></figure>
|
|
<p>比较重要的是这里的 static 代码块还有一个逻辑，就是下面这个初始化逻辑</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">static</span> {</span><br><span class="line"> SINGLETON.init();</span><br><span class="line">}</span><br><span class="line"><span class="keyword">void</span> <span class="title function_">init</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">ContextInitializer</span>(defaultLoggerContext).autoConfig();</span><br><span class="line"> } <span class="keyword">catch</span> (JoranException je) {</span><br><span class="line"> Util.report(<span class="string">"Failed to auto configure default logger context"</span>, je);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// logback-292</span></span><br><span class="line"> <span class="keyword">if</span> (!StatusUtil.contextHasStatusListener(defaultLoggerContext)) {</span><br><span class="line"> StatusPrinter.printInCaseOfErrorsOrWarnings(defaultLoggerContext);</span><br><span class="line"> }</span><br><span class="line"> contextSelectorBinder.init(defaultLoggerContext, KEY);</span><br><span class="line"> initialized = <span class="literal">true</span>;</span><br><span class="line"> } <span class="keyword">catch</span> (Exception t) { <span class="comment">// see LOGBACK-1159</span></span><br><span class="line"> Util.report(<span class="string">"Failed to instantiate ["</span> + LoggerContext.class.getName() + <span class="string">"]"</span>, t);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>上面的 defaultLoggerContext 也是通过静态代码块处理的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="type">LoggerContext</span> <span class="variable">defaultLoggerContext</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">LoggerContext</span>();</span><br></pre></td></tr></table></figure>
|
|
<p>然后是调用 <code>ch.qos.logback.classic.util.ContextInitializer#autoConfig</code> 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">autoConfig</span><span class="params">()</span> <span class="keyword">throws</span> JoranException {</span><br><span class="line"> StatusListenerConfigHelper.installIfAsked(loggerContext);</span><br><span class="line"> <span class="type">URL</span> <span class="variable">url</span> <span class="operator">=</span> findURLOfDefaultConfigurationFile(<span class="literal">true</span>);</span><br><span class="line"> <span class="keyword">if</span> (url != <span class="literal">null</span>) {</span><br><span class="line"> configureByResource(url);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="type">Configurator</span> <span class="variable">c</span> <span class="operator">=</span> EnvUtil.loadFromServiceLoader(Configurator.class);</span><br><span class="line"> <span class="keyword">if</span> (c != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> c.setContext(loggerContext);</span><br><span class="line"> c.configure(loggerContext);</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">LogbackException</span>(String.format(<span class="string">"Failed to initialize Configurator: %s using ServiceLoader"</span>, c != <span class="literal">null</span> ? c.getClass()</span><br><span class="line"> .getCanonicalName() : <span class="string">"null"</span>), e);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="type">BasicConfigurator</span> <span class="variable">basicConfigurator</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">BasicConfigurator</span>();</span><br><span class="line"> basicConfigurator.setContext(loggerContext);</span><br><span class="line"> basicConfigurator.configure(loggerContext);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>前面会先查找配置文件路径<br><code>URL url = findURLOfDefaultConfigurationFile(true);</code> 还是调用这个类内的 <code>ch.qos.logback.classic.util.ContextInitializer#findURLOfDefaultConfigurationFile</code> 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> URL <span class="title function_">findURLOfDefaultConfigurationFile</span><span class="params">(<span class="type">boolean</span> updateStatus)</span> {</span><br><span class="line"> <span class="type">ClassLoader</span> <span class="variable">myClassLoader</span> <span class="operator">=</span> Loader.getClassLoaderOfObject(<span class="built_in">this</span>);</span><br><span class="line"> <span class="type">URL</span> <span class="variable">url</span> <span class="operator">=</span> findConfigFileURLFromSystemProperties(myClassLoader, updateStatus);</span><br><span class="line"> <span class="keyword">if</span> (url != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> url;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> url = getResource(TEST_AUTOCONFIG_FILE, myClassLoader, updateStatus);</span><br><span class="line"> <span class="keyword">if</span> (url != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> url;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> url = getResource(GROOVY_AUTOCONFIG_FILE, myClassLoader, updateStatus);</span><br><span class="line"> <span class="keyword">if</span> (url != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> url;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> getResource(AUTOCONFIG_FILE, myClassLoader, updateStatus);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>才发现原来这里也是硬编码了文件名</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">ContextInitializer</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">final</span> <span class="keyword">public</span> <span class="keyword">static</span> <span class="type">String</span> <span class="variable">GROOVY_AUTOCONFIG_FILE</span> <span class="operator">=</span> <span class="string">"logback.groovy"</span>;</span><br><span class="line"> <span class="keyword">final</span> <span class="keyword">public</span> <span class="keyword">static</span> <span class="type">String</span> <span class="variable">AUTOCONFIG_FILE</span> <span class="operator">=</span> <span class="string">"logback.xml"</span>;</span><br><span class="line"> <span class="keyword">final</span> <span class="keyword">public</span> <span class="keyword">static</span> <span class="type">String</span> <span class="variable">TEST_AUTOCONFIG_FILE</span> <span class="operator">=</span> <span class="string">"logback-test.xml"</span>;</span><br><span class="line"> <span class="keyword">final</span> <span class="keyword">public</span> <span class="keyword">static</span> <span class="type">String</span> <span class="variable">CONFIG_FILE_PROPERTY</span> <span class="operator">=</span> <span class="string">"logback.configurationFile"</span>;</span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
<p>如果不为空就调用了<br><code>ch.qos.logback.classic.util.ContextInitializer#configureByResource</code> 这边是根据后缀判断处理方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">configureByResource</span><span class="params">(URL url)</span> <span class="keyword">throws</span> JoranException {</span><br><span class="line"> <span class="keyword">if</span> (url == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalArgumentException</span>(<span class="string">"URL argument cannot be null"</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">urlString</span> <span class="operator">=</span> url.toString();</span><br><span class="line"> <span class="keyword">if</span> (urlString.endsWith(<span class="string">"groovy"</span>)) {</span><br><span class="line"> <span class="keyword">if</span> (EnvUtil.isGroovyAvailable()) {</span><br><span class="line"> <span class="comment">// avoid directly referring to GafferConfigurator so as to avoid</span></span><br><span class="line"> <span class="comment">// loading groovy.lang.GroovyObject . See also http://jira.qos.ch/browse/LBCLASSIC-214</span></span><br><span class="line"> GafferUtil.runGafferConfiguratorOn(loggerContext, <span class="built_in">this</span>, url);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="type">StatusManager</span> <span class="variable">sm</span> <span class="operator">=</span> loggerContext.getStatusManager();</span><br><span class="line"> sm.add(<span class="keyword">new</span> <span class="title class_">ErrorStatus</span>(<span class="string">"Groovy classes are not available on the class path. ABORTING INITIALIZATION."</span>, loggerContext));</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (urlString.endsWith(<span class="string">"xml"</span>)) {</span><br><span class="line"> <span class="type">JoranConfigurator</span> <span class="variable">configurator</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">JoranConfigurator</span>();</span><br><span class="line"> configurator.setContext(loggerContext);</span><br><span class="line"> configurator.doConfigure(url);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">LogbackException</span>(<span class="string">"Unexpected filename extension of file ["</span> + url.toString() + <span class="string">"]. Should be either .groovy or .xml"</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>因为我们找到的是 logback.xml，所以走的是<br><code>ch.qos.logback.classic.joran.JoranConfigurator</code> ，<br>然后调用了<br><code>ch.qos.logback.core.joran.GenericConfigurator#doConfigure(java.net.URL)</code><br>这个 <code>GenericConfigurator</code> 是 <code>JoranConfigurator</code> 的父类</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">final</span> <span class="keyword">void</span> <span class="title function_">doConfigure</span><span class="params">(URL url)</span> <span class="keyword">throws</span> JoranException {</span><br><span class="line"> <span class="type">InputStream</span> <span class="variable">in</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> informContextOfURLUsedForConfiguration(getContext(), url);</span><br><span class="line"> <span class="type">URLConnection</span> <span class="variable">urlConnection</span> <span class="operator">=</span> url.openConnection();</span><br><span class="line"> <span class="comment">// per http://jira.qos.ch/browse/LBCORE-105</span></span><br><span class="line"> <span class="comment">// per http://jira.qos.ch/browse/LBCORE-127</span></span><br><span class="line"> urlConnection.setUseCaches(<span class="literal">false</span>);</span><br><span class="line"></span><br><span class="line"> in = urlConnection.getInputStream();</span><br><span class="line"> doConfigure(in, url.toExternalForm());</span><br><span class="line"> } <span class="keyword">catch</span> (IOException ioe) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">errMsg</span> <span class="operator">=</span> <span class="string">"Could not open URL ["</span> + url + <span class="string">"]."</span>;</span><br><span class="line"> addError(errMsg, ioe);</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">JoranException</span>(errMsg, ioe);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="keyword">if</span> (in != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> in.close();</span><br><span class="line"> } <span class="keyword">catch</span> (IOException ioe) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">errMsg</span> <span class="operator">=</span> <span class="string">"Could not close input stream"</span>;</span><br><span class="line"> addError(errMsg, ioe);</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">JoranException</span>(errMsg, ioe);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后继续调用<br><code>ch.qos.logback.core.joran.GenericConfigurator#doConfigure(java.io.InputStream, java.lang.String)</code> 处理具体的内容，前面处理了内容获取，因为如果是走的 url 文件就需要从网络获取文件流，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">final</span> <span class="keyword">void</span> <span class="title function_">doConfigure</span><span class="params">(InputStream inputStream, String systemId)</span> <span class="keyword">throws</span> JoranException {</span><br><span class="line"> <span class="type">InputSource</span> <span class="variable">inputSource</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">InputSource</span>(inputStream);</span><br><span class="line"> inputSource.setSystemId(systemId);</span><br><span class="line"> doConfigure(inputSource);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>而实际的处理 xml 内容则是在<br><code>ch.qos.logback.core.joran.GenericConfigurator#doConfigure(org.xml.sax.InputSource)</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">final</span> <span class="keyword">void</span> <span class="title function_">doConfigure</span><span class="params">(<span class="keyword">final</span> InputSource inputSource)</span> <span class="keyword">throws</span> JoranException {</span><br><span class="line"></span><br><span class="line"> <span class="type">long</span> <span class="variable">threshold</span> <span class="operator">=</span> System.currentTimeMillis();</span><br><span class="line"> <span class="comment">// if (!ConfigurationWatchListUtil.wasConfigurationWatchListReset(context)) {</span></span><br><span class="line"> <span class="comment">// informContextOfURLUsedForConfiguration(getContext(), null);</span></span><br><span class="line"> <span class="comment">// }</span></span><br><span class="line"> <span class="type">SaxEventRecorder</span> <span class="variable">recorder</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SaxEventRecorder</span>(context);</span><br><span class="line"> recorder.recordEvents(inputSource);</span><br><span class="line"> doConfigure(recorder.saxEventList);</span><br><span class="line"> <span class="comment">// no exceptions a this level</span></span><br><span class="line"> <span class="type">StatusUtil</span> <span class="variable">statusUtil</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">StatusUtil</span>(context);</span><br><span class="line"> <span class="keyword">if</span> (statusUtil.noXMLParsingErrorsOccurred(threshold)) {</span><br><span class="line"> addInfo(<span class="string">"Registering current configuration as safe fallback point"</span>);</span><br><span class="line"> registerSafeConfiguration(recorder.saxEventList);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>需要解析 xml 了，先是调用 <code>buildInterpreter</code> 构建了 Interpreter </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">doConfigure</span><span class="params">(<span class="keyword">final</span> List<SaxEvent> eventList)</span> <span class="keyword">throws</span> JoranException {</span><br><span class="line"> buildInterpreter();</span><br><span class="line"> <span class="comment">// disallow simultaneous configurations of the same context</span></span><br><span class="line"> <span class="keyword">synchronized</span> (context.getConfigurationLock()) {</span><br><span class="line"> interpreter.getEventPlayer().play(eventList);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>build 里还有一些逻辑，就是匹配后面要处理的规则集，就是要确认那些标签要处理</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">buildInterpreter</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">RuleStore</span> <span class="variable">rs</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SimpleRuleStore</span>(context);</span><br><span class="line"> addInstanceRules(rs);</span><br><span class="line"> <span class="built_in">this</span>.interpreter = <span class="keyword">new</span> <span class="title class_">Interpreter</span>(context, rs, initialElementPath());</span><br><span class="line"> <span class="type">InterpretationContext</span> <span class="variable">interpretationContext</span> <span class="operator">=</span> interpreter.getInterpretationContext();</span><br><span class="line"> interpretationContext.setContext(context);</span><br><span class="line"> addImplicitRules(interpreter);</span><br><span class="line"> addDefaultNestedComponentRegistryRules(interpretationContext.getDefaultNestedComponentRegistry());</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>第一步就是先 new 一个 <code>ch.qos.logback.core.joran.spi.SimpleRuleStore</code> 然后调用了 <code>ch.qos.logback.classic.joran.JoranConfigurator#addInstanceRules</code> 来添加规则，就是下面的，可以看到对于 logger,appender 这些的配置，其中前面点的父类也又一些规则 ,<code>ch.qos.logback.core.joran.JoranConfiguratorBase</code> 逻辑类似就不展开了，这里的规则就对应了后面的各种 Action，就是对应的处理逻辑</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">addInstanceRules</span><span class="params">(RuleStore rs)</span> {</span><br><span class="line"> <span class="comment">// parent rules already added</span></span><br><span class="line"> <span class="built_in">super</span>.addInstanceRules(rs);</span><br><span class="line"></span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"configuration"</span>), <span class="keyword">new</span> <span class="title class_">ConfigurationAction</span>());</span><br><span class="line"></span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"configuration/contextName"</span>), <span class="keyword">new</span> <span class="title class_">ContextNameAction</span>());</span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"configuration/contextListener"</span>), <span class="keyword">new</span> <span class="title class_">LoggerContextListenerAction</span>());</span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"configuration/insertFromJNDI"</span>), <span class="keyword">new</span> <span class="title class_">InsertFromJNDIAction</span>());</span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"configuration/evaluator"</span>), <span class="keyword">new</span> <span class="title class_">EvaluatorAction</span>());</span><br><span class="line"></span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"configuration/appender/sift"</span>), <span class="keyword">new</span> <span class="title class_">SiftAction</span>());</span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"configuration/appender/sift/*"</span>), <span class="keyword">new</span> <span class="title class_">NOPAction</span>());</span><br><span class="line"></span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"configuration/logger"</span>), <span class="keyword">new</span> <span class="title class_">LoggerAction</span>());</span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"configuration/logger/level"</span>), <span class="keyword">new</span> <span class="title class_">LevelAction</span>());</span><br><span class="line"></span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"configuration/root"</span>), <span class="keyword">new</span> <span class="title class_">RootLoggerAction</span>());</span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"configuration/root/level"</span>), <span class="keyword">new</span> <span class="title class_">LevelAction</span>());</span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"configuration/logger/appender-ref"</span>), <span class="keyword">new</span> <span class="title class_">AppenderRefAction</span><ILoggingEvent>());</span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"configuration/root/appender-ref"</span>), <span class="keyword">new</span> <span class="title class_">AppenderRefAction</span><ILoggingEvent>());</span><br><span class="line"></span><br><span class="line"> <span class="comment">// add if-then-else support</span></span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"*/if"</span>), <span class="keyword">new</span> <span class="title class_">IfAction</span>());</span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"*/if/then"</span>), <span class="keyword">new</span> <span class="title class_">ThenAction</span>());</span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"*/if/then/*"</span>), <span class="keyword">new</span> <span class="title class_">NOPAction</span>());</span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"*/if/else"</span>), <span class="keyword">new</span> <span class="title class_">ElseAction</span>());</span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"*/if/else/*"</span>), <span class="keyword">new</span> <span class="title class_">NOPAction</span>());</span><br><span class="line"></span><br><span class="line"> <span class="comment">// add jmxConfigurator only if we have JMX available.</span></span><br><span class="line"> <span class="comment">// If running under JDK 1.4 (retrotranslateed logback) then we</span></span><br><span class="line"> <span class="comment">// might not have JMX.</span></span><br><span class="line"> <span class="keyword">if</span> (PlatformInfo.hasJMXObjectName()) {</span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"configuration/jmxConfigurator"</span>), <span class="keyword">new</span> <span class="title class_">JMXConfiguratorAction</span>());</span><br><span class="line"> }</span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"configuration/include"</span>), <span class="keyword">new</span> <span class="title class_">IncludeAction</span>());</span><br><span class="line"></span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"configuration/consolePlugin"</span>), <span class="keyword">new</span> <span class="title class_">ConsolePluginAction</span>());</span><br><span class="line"></span><br><span class="line"> rs.addRule(<span class="keyword">new</span> <span class="title class_">ElementSelector</span>(<span class="string">"configuration/receiver"</span>), <span class="keyword">new</span> <span class="title class_">ReceiverAction</span>());</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后调用了 interpreter 的 eventPlayer 处理 xml 的各个标签</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">play</span><span class="params">(List<SaxEvent> aSaxEventList)</span> {</span><br><span class="line"> eventList = aSaxEventList;</span><br><span class="line"> SaxEvent se;</span><br><span class="line"> <span class="keyword">for</span> (currentIndex = <span class="number">0</span>; currentIndex < eventList.size(); currentIndex++) {</span><br><span class="line"> se = eventList.get(currentIndex);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (se <span class="keyword">instanceof</span> StartEvent) {</span><br><span class="line"> interpreter.startElement((StartEvent) se);</span><br><span class="line"> <span class="comment">// invoke fireInPlay after startElement processing</span></span><br><span class="line"> interpreter.getInterpretationContext().fireInPlay(se);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (se <span class="keyword">instanceof</span> BodyEvent) {</span><br><span class="line"> <span class="comment">// invoke fireInPlay before characters processing</span></span><br><span class="line"> interpreter.getInterpretationContext().fireInPlay(se);</span><br><span class="line"> interpreter.characters((BodyEvent) se);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (se <span class="keyword">instanceof</span> EndEvent) {</span><br><span class="line"> <span class="comment">// invoke fireInPlay before endElement processing</span></span><br><span class="line"> interpreter.getInterpretationContext().fireInPlay(se);</span><br><span class="line"> interpreter.endElement((EndEvent) se);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>接下去就在 <code>startElement</code> 中调用会先获取 <code>getApplicableActionList</code> 适配的 Action， <code>callBeginAction</code> </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">startElement</span><span class="params">(String namespaceURI, String localName, String qName, Attributes atts)</span> {</span><br><span class="line"></span><br><span class="line"> <span class="type">String</span> <span class="variable">tagName</span> <span class="operator">=</span> getTagName(localName, qName);</span><br><span class="line"> elementPath.push(tagName);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (skip != <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// every startElement pushes an action list</span></span><br><span class="line"> pushEmptyActionList();</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> List<Action> applicableActionList = getApplicableActionList(elementPath, atts);</span><br><span class="line"> <span class="keyword">if</span> (applicableActionList != <span class="literal">null</span>) {</span><br><span class="line"> actionListStack.add(applicableActionList);</span><br><span class="line"> callBeginAction(applicableActionList, tagName, atts);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// every startElement pushes an action list</span></span><br><span class="line"> pushEmptyActionList();</span><br><span class="line"> <span class="type">String</span> <span class="variable">errMsg</span> <span class="operator">=</span> <span class="string">"no applicable action for ["</span> + tagName + <span class="string">"], current ElementPath is ["</span> + elementPath + <span class="string">"]"</span>;</span><br><span class="line"> cai.addError(errMsg);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>里面就是用了 ruleStore 来判断是否适配，并且取出对应的 Action</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">List<Action> <span class="title function_">getApplicableActionList</span><span class="params">(ElementPath elementPath, Attributes attributes)</span> {</span><br><span class="line"> List<Action> applicableActionList = ruleStore.matchActions(elementPath);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// logger.debug("set of applicable patterns: " + applicableActionList);</span></span><br><span class="line"> <span class="keyword">if</span> (applicableActionList == <span class="literal">null</span>) {</span><br><span class="line"> applicableActionList = lookupImplicitAction(elementPath, attributes, interpretationContext);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> applicableActionList;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>比如我这边往下处理，通过规则匹配拿到了 LoggerAction, 然后调用 <code>Action.begin</code> 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">void</span> <span class="title function_">callBeginAction</span><span class="params">(List<Action> applicableActionList, String tagName, Attributes atts)</span> {</span><br><span class="line"> <span class="keyword">if</span> (applicableActionList == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> Iterator<Action> i = applicableActionList.iterator();</span><br><span class="line"> <span class="keyword">while</span> (i.hasNext()) {</span><br><span class="line"> <span class="type">Action</span> <span class="variable">action</span> <span class="operator">=</span> (Action) i.next();</span><br><span class="line"> <span class="comment">// now let us invoke the action. We catch and report any eventual</span></span><br><span class="line"> <span class="comment">// exceptions</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> action.begin(interpretationContext, tagName, atts);</span><br><span class="line"> } <span class="keyword">catch</span> (ActionException e) {</span><br><span class="line"> skip = elementPath.duplicate();</span><br><span class="line"> cai.addError(<span class="string">"ActionException in Action for tag ["</span> + tagName + <span class="string">"]"</span>, e);</span><br><span class="line"> } <span class="keyword">catch</span> (RuntimeException e) {</span><br><span class="line"> skip = elementPath.duplicate();</span><br><span class="line"> cai.addError(<span class="string">"RuntimeException in Action for tag ["</span> + tagName + <span class="string">"]"</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>就是下面这个</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">begin</span><span class="params">(InterpretationContext ec, String name, Attributes attributes)</span> {</span><br><span class="line"> <span class="comment">// Let us forget about previous errors (in this object)</span></span><br><span class="line"> inError = <span class="literal">false</span>;</span><br><span class="line"> logger = <span class="literal">null</span>;</span><br><span class="line"></span><br><span class="line"> <span class="type">LoggerContext</span> <span class="variable">loggerContext</span> <span class="operator">=</span> (LoggerContext) <span class="built_in">this</span>.context;</span><br><span class="line"></span><br><span class="line"> <span class="type">String</span> <span class="variable">loggerName</span> <span class="operator">=</span> ec.subst(attributes.getValue(NAME_ATTRIBUTE));</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (OptionHelper.isEmpty(loggerName)) {</span><br><span class="line"> inError = <span class="literal">true</span>;</span><br><span class="line"> <span class="type">String</span> <span class="variable">aroundLine</span> <span class="operator">=</span> getLineColStr(ec);</span><br><span class="line"> <span class="type">String</span> <span class="variable">errorMsg</span> <span class="operator">=</span> <span class="string">"No 'name' attribute in element "</span> + name + <span class="string">", around "</span> + aroundLine;</span><br><span class="line"> addError(errorMsg);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> logger = loggerContext.getLogger(loggerName);</span><br><span class="line"></span><br><span class="line"> <span class="type">String</span> <span class="variable">levelStr</span> <span class="operator">=</span> ec.subst(attributes.getValue(LEVEL_ATTRIBUTE));</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!OptionHelper.isEmpty(levelStr)) {</span><br><span class="line"> <span class="keyword">if</span> (ActionConst.INHERITED.equalsIgnoreCase(levelStr) || ActionConst.NULL.equalsIgnoreCase(levelStr)) {</span><br><span class="line"> addInfo(<span class="string">"Setting level of logger ["</span> + loggerName + <span class="string">"] to null, i.e. INHERITED"</span>);</span><br><span class="line"> logger.setLevel(<span class="literal">null</span>);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="type">Level</span> <span class="variable">level</span> <span class="operator">=</span> Level.toLevel(levelStr);</span><br><span class="line"> addInfo(<span class="string">"Setting level of logger ["</span> + loggerName + <span class="string">"] to "</span> + level);</span><br><span class="line"> logger.setLevel(level);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">String</span> <span class="variable">additivityStr</span> <span class="operator">=</span> ec.subst(attributes.getValue(ActionConst.ADDITIVITY_ATTRIBUTE));</span><br><span class="line"> <span class="keyword">if</span> (!OptionHelper.isEmpty(additivityStr)) {</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">additive</span> <span class="operator">=</span> OptionHelper.toBoolean(additivityStr, <span class="literal">true</span>);</span><br><span class="line"> addInfo(<span class="string">"Setting additivity of logger ["</span> + loggerName + <span class="string">"] to "</span> + additive);</span><br><span class="line"> logger.setAdditive(additive);</span><br><span class="line"> }</span><br><span class="line"> ec.pushObject(logger);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里就会处理 logger 获取逻辑</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">final</span> Logger <span class="title function_">getLogger</span><span class="params">(<span class="keyword">final</span> String name)</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (name == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalArgumentException</span>(<span class="string">"name argument cannot be null"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// if we are asking for the root logger, then let us return it without</span></span><br><span class="line"> <span class="comment">// wasting time</span></span><br><span class="line"> <span class="keyword">if</span> (Logger.ROOT_LOGGER_NAME.equalsIgnoreCase(name)) {</span><br><span class="line"> <span class="keyword">return</span> root;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="type">Logger</span> <span class="variable">logger</span> <span class="operator">=</span> root;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// check if the desired logger exists, if it does, return it</span></span><br><span class="line"> <span class="comment">// without further ado.</span></span><br><span class="line"> <span class="type">Logger</span> <span class="variable">childLogger</span> <span class="operator">=</span> (Logger) loggerCache.get(name);</span><br><span class="line"> <span class="comment">// if we have the child, then let us return it without wasting time</span></span><br><span class="line"> <span class="keyword">if</span> (childLogger != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> childLogger;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// if the desired logger does not exist, them create all the loggers</span></span><br><span class="line"> <span class="comment">// in between as well (if they don't already exist)</span></span><br><span class="line"> String childName;</span><br><span class="line"> <span class="keyword">while</span> (<span class="literal">true</span>) {</span><br><span class="line"> <span class="type">int</span> <span class="variable">h</span> <span class="operator">=</span> LoggerNameUtil.getSeparatorIndexOf(name, i);</span><br><span class="line"> <span class="keyword">if</span> (h == -<span class="number">1</span>) {</span><br><span class="line"> childName = name;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> childName = name.substring(<span class="number">0</span>, h);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// move i left of the last point</span></span><br><span class="line"> i = h + <span class="number">1</span>;</span><br><span class="line"> <span class="keyword">synchronized</span> (logger) {</span><br><span class="line"> childLogger = logger.getChildByName(childName);</span><br><span class="line"> <span class="keyword">if</span> (childLogger == <span class="literal">null</span>) {</span><br><span class="line"> childLogger = logger.createChildByName(childName);</span><br><span class="line"> loggerCache.put(childName, childLogger);</span><br><span class="line"> incSize();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> logger = childLogger;</span><br><span class="line"> <span class="keyword">if</span> (h == -<span class="number">1</span>) {</span><br><span class="line"> <span class="keyword">return</span> childLogger;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>如果从 <code>loggerCache</code> 中能取到就直接获取，如果不能就会去创建</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊一下 RocketMQ 的消息存储之 MMAP</title>
|
|
<url>/2021/09/04/%E8%81%8A%E4%B8%80%E4%B8%8B-RocketMQ-%E7%9A%84%E6%B6%88%E6%81%AF%E5%AD%98%E5%82%A8/</url>
|
|
<content><![CDATA[<p>这是个很大的话题了，可能会分成两部分说，第一部分就是所谓的零拷贝 ( zero-copy )，这一块其实也不新鲜，我对零拷贝的概念主要来自<a href="https://www.linuxjournal.com/article/6345">这篇文章</a>，个人感觉写得非常好，在 rocketmq 中，最大的一块存储就是消息存储，也就是 CommitLog ，当然还有 ConsumeQueue 和 IndexFile，以及其他一些文件，CommitLog 的存储是以一个 1G 大小的文件作为存储单位，写完了就再建一个，那么如何提高这 1G 文件的读写效率呢，就是 mmap，传统意义的读写文件，read，write 都需要由系统调用，来回地在用户态跟内核态进行拷贝切换，</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line">read(file, tmp_buf, len);</span><br><span class="line">write(socket, tmp_buf, len);</span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
|
|
<p><img data-src="https://img.nicksxs.me/uPic/vms95Z.jpg" alt="vms95Z"></p>
|
|
<p>如上面的图显示的，要在用户态跟内核态进行切换，数据还需要在内核缓冲跟用户缓冲之间拷贝多次，</p>
|
|
<blockquote>
|
|
<ol>
|
|
<li>第一步是调用 read，需要在用户态切换成内核态，<a href="https://en.wikipedia.org/wiki/Direct_memory_access">DMA</a>模块从磁盘中读取文件，并存储在内核缓冲区，相当于是第一次复制</li>
|
|
<li>数据从内核缓冲区被拷贝到用户缓冲区，read 调用返回，伴随着内核态又切换成用户态，完成了第二次复制</li>
|
|
<li>然后是write 写入，这里也会伴随着用户态跟内核态的切换，数据从用户缓冲区被复制到内核空间缓冲区，完成了第三次复制，这次有点不一样的是数据不是在内核缓冲区了，会复制到 socket buffer 中。</li>
|
|
<li>write 系统调用返回，又切换回了用户态，然后数据由 DMA 拷贝到协议引擎。</li>
|
|
</ol>
|
|
</blockquote>
|
|
<p>如此就能看出其实默认的读写操作代价是非常大的，而在 rocketmq 等高性能中间件中都有使用的零拷贝技术，其中 rocketmq 使用的是 mmap</p>
|
|
<h3 id="mmap"><a href="#mmap" class="headerlink" title="mmap"></a>mmap</h3><p>mmap基于 OS 的 <a href="https://en.wikipedia.org/wiki/Mmap">mmap</a> 的内存映射技术，通过<a href="https://en.wikipedia.org/wiki/Memory_management_unit">MMU</a> 映射文件，将文件直接映射到用户态的内存地址，使得对文件的操作不再是 write/read,而转化为直接对内存地址的操作，使随机读写文件和读写内存相似的速度。</p>
|
|
<blockquote>
|
|
<p>mmap 把文件映射到用户空间里的虚拟内存，省去了从内核缓冲区复制到用户空间的过程，文件中的位置在虚拟内存中有了对应的地址，可以像操作内存一样操作这个文件，这样的文件读写文件方式少了数据从内核缓存到用户空间的拷贝，效率很高。</p>
|
|
</blockquote>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line">tmp_buf = mmap(file, len);</span><br><span class="line">write(socket, tmp_buf, len);</span><br></pre></td></tr></table></figure>
|
|
|
|
<p><img data-src="https://img.nicksxs.me/uPic/I68mFx.jpg" alt="I68mFx"></p>
|
|
<blockquote>
|
|
<p>第一步：mmap系统调用使得文件内容被DMA引擎复制到内核缓冲区。然后该缓冲区与用户进程共享，在内核和用户内存空间之间不进行任何拷贝。</p>
|
|
<p>第二步：写系统调用使得内核将数据从原来的内核缓冲区复制到与套接字相关的内核缓冲区。</p>
|
|
<p>第三步：第三次拷贝发生在DMA引擎将数据从内核套接字缓冲区传递给协议引擎时。</p>
|
|
<p>通过使用mmap而不是read，我们将内核需要拷贝的数据量减少了一半。当大量的数据被传输时，这将有很好的效果。然而，这种改进并不是没有代价的；在使用mmap+write方法时，有一些隐藏的陷阱。例如当你对一个文件进行内存映射，然后在另一个进程截断同一文件时调用写。你的写系统调用将被总线错误信号SIGBUS打断，因为你执行了一个错误的内存访问。该信号的默认行为是杀死进程并dumpcore–这对网络服务器来说不是最理想的操作。</p>
|
|
<p>有两种方法可以解决这个问题。</p>
|
|
<p>第一种方法是为SIGBUS信号安装一个信号处理程序，然后在处理程序中简单地调用返回。通过这样做，写系统调用会返回它在被打断之前所写的字节数，并将errno设置为成功。让我指出，这将是一个糟糕的解决方案，一个治标不治本的解决方案。因为SIGBUS预示着进程出了严重的问题，所以不鼓励使用这种解决方案。</p>
|
|
<p>第二个解决方案涉及内核的文件租赁（在Windows中称为 “机会锁”）。这是解决这个问题的正确方法。通过在文件描述符上使用租赁，你与内核在一个特定的文件上达成租约。然后你可以向内核请求一个读/写租约。当另一个进程试图截断你正在传输的文件时，内核会向你发送一个实时信号，即RT_SIGNAL_LEASE信号。它告诉你内核即将终止你对该文件的写或读租约。在你的程序访问一个无效的地址和被SIGBUS信号杀死之前，你的写调用会被打断了。写入调用的返回值是中断前写入的字节数，errno将被设置为成功。下面是一些示例代码，显示了如何从内核中获得租约。</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="keyword">if</span>(fcntl(fd, F_SETSIG, RT_SIGNAL_LEASE) == <span class="number">-1</span>) {</span><br><span class="line"> perror(<span class="string">"kernel lease set signal"</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">}</span><br><span class="line"><span class="comment">/* l_type can be F_RDLCK F_WRLCK */</span></span><br><span class="line"><span class="keyword">if</span>(fcntl(fd, F_SETLEASE, l_type)){</span><br><span class="line"> perror(<span class="string">"kernel lease set type"</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure></blockquote>
|
|
]]></content>
|
|
<categories>
|
|
<category>MQ</category>
|
|
<category>RocketMQ</category>
|
|
<category>消息队列</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>MQ</tag>
|
|
<tag>消息队列</tag>
|
|
<tag>RocketMQ</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊一下 RocketMQ 的 NameServer 源码</title>
|
|
<url>/2020/07/05/%E8%81%8A%E4%B8%80%E4%B8%8B-RocketMQ-%E7%9A%84-NameServer-%E6%BA%90%E7%A0%81/</url>
|
|
<content><![CDATA[<p>前面介绍了，nameserver相当于dubbo的注册中心，用与管理broker，broker会在启动的时候注册到nameserver，并且会发送心跳给namaserver，nameserver负责保存活跃的broker，包括master和slave，同时保存topic和topic下的队列，以及filter列表，然后为producer和consumer的请求提供服务。</p>
|
|
<h3 id="启动过程"><a href="#启动过程" class="headerlink" title="启动过程"></a>启动过程</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> main0(args);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> NamesrvController <span class="title function_">main0</span><span class="params">(String[] args)</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">NamesrvController</span> <span class="variable">controller</span> <span class="operator">=</span> createNamesrvController(args);</span><br><span class="line"> start(controller);</span><br><span class="line"> <span class="type">String</span> <span class="variable">tip</span> <span class="operator">=</span> <span class="string">"The Name Server boot success. serializeType="</span> + RemotingCommand.getSerializeTypeConfigInThisServer();</span><br><span class="line"> log.info(tip);</span><br><span class="line"> System.out.printf(<span class="string">"%s%n"</span>, tip);</span><br><span class="line"> <span class="keyword">return</span> controller;</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> System.exit(-<span class="number">1</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>入口的代码时这样子，其实主要的逻辑在createNamesrvController和start方法，来看下这两个的实现</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> NamesrvController <span class="title function_">createNamesrvController</span><span class="params">(String[] args)</span> <span class="keyword">throws</span> IOException, JoranException {</span><br><span class="line"> System.setProperty(RemotingCommand.REMOTING_VERSION_KEY, Integer.toString(MQVersion.CURRENT_VERSION));</span><br><span class="line"> <span class="comment">//PackageConflictDetect.detectFastjson();</span></span><br><span class="line"></span><br><span class="line"> <span class="type">Options</span> <span class="variable">options</span> <span class="operator">=</span> ServerUtil.buildCommandlineOptions(<span class="keyword">new</span> <span class="title class_">Options</span>());</span><br><span class="line"> commandLine = ServerUtil.parseCmdLine(<span class="string">"mqnamesrv"</span>, args, buildCommandlineOptions(options), <span class="keyword">new</span> <span class="title class_">PosixParser</span>());</span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == commandLine) {</span><br><span class="line"> System.exit(-<span class="number">1</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">final</span> <span class="type">NamesrvConfig</span> <span class="variable">namesrvConfig</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">NamesrvConfig</span>();</span><br><span class="line"> <span class="keyword">final</span> <span class="type">NettyServerConfig</span> <span class="variable">nettyServerConfig</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">NettyServerConfig</span>();</span><br><span class="line"> nettyServerConfig.setListenPort(<span class="number">9876</span>);</span><br><span class="line"> <span class="keyword">if</span> (commandLine.hasOption(<span class="string">'c'</span>)) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">file</span> <span class="operator">=</span> commandLine.getOptionValue(<span class="string">'c'</span>);</span><br><span class="line"> <span class="keyword">if</span> (file != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">InputStream</span> <span class="variable">in</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">BufferedInputStream</span>(<span class="keyword">new</span> <span class="title class_">FileInputStream</span>(file));</span><br><span class="line"> properties = <span class="keyword">new</span> <span class="title class_">Properties</span>();</span><br><span class="line"> properties.load(in);</span><br><span class="line"> MixAll.properties2Object(properties, namesrvConfig);</span><br><span class="line"> MixAll.properties2Object(properties, nettyServerConfig);</span><br><span class="line"></span><br><span class="line"> namesrvConfig.setConfigStorePath(file);</span><br><span class="line"></span><br><span class="line"> System.out.printf(<span class="string">"load config properties file OK, %s%n"</span>, file);</span><br><span class="line"> in.close();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (commandLine.hasOption(<span class="string">'p'</span>)) {</span><br><span class="line"> <span class="type">InternalLogger</span> <span class="variable">console</span> <span class="operator">=</span> InternalLoggerFactory.getLogger(LoggerName.NAMESRV_CONSOLE_NAME);</span><br><span class="line"> MixAll.printObjectProperties(console, namesrvConfig);</span><br><span class="line"> MixAll.printObjectProperties(console, nettyServerConfig);</span><br><span class="line"> System.exit(<span class="number">0</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> MixAll.properties2Object(ServerUtil.commandLine2Properties(commandLine), namesrvConfig);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == namesrvConfig.getRocketmqHome()) {</span><br><span class="line"> System.out.printf(<span class="string">"Please set the %s variable in your environment to match the location of the RocketMQ installation%n"</span>, MixAll.ROCKETMQ_HOME_ENV);</span><br><span class="line"> System.exit(-<span class="number">2</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">LoggerContext</span> <span class="variable">lc</span> <span class="operator">=</span> (LoggerContext) LoggerFactory.getILoggerFactory();</span><br><span class="line"> <span class="type">JoranConfigurator</span> <span class="variable">configurator</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">JoranConfigurator</span>();</span><br><span class="line"> configurator.setContext(lc);</span><br><span class="line"> lc.reset();</span><br><span class="line"> configurator.doConfigure(namesrvConfig.getRocketmqHome() + <span class="string">"/conf/logback_namesrv.xml"</span>);</span><br><span class="line"></span><br><span class="line"> log = InternalLoggerFactory.getLogger(LoggerName.NAMESRV_LOGGER_NAME);</span><br><span class="line"></span><br><span class="line"> MixAll.printObjectProperties(log, namesrvConfig);</span><br><span class="line"> MixAll.printObjectProperties(log, nettyServerConfig);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">final</span> <span class="type">NamesrvController</span> <span class="variable">controller</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">NamesrvController</span>(namesrvConfig, nettyServerConfig);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// remember all configs to prevent discard</span></span><br><span class="line"> controller.getConfiguration().registerConfig(properties);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> controller;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这个方法里其实主要是读取一些配置啥的，不是很复杂，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> NamesrvController <span class="title function_">start</span><span class="params">(<span class="keyword">final</span> NamesrvController controller)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == controller) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalArgumentException</span>(<span class="string">"NamesrvController is null"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">boolean</span> <span class="variable">initResult</span> <span class="operator">=</span> controller.initialize();</span><br><span class="line"> <span class="keyword">if</span> (!initResult) {</span><br><span class="line"> controller.shutdown();</span><br><span class="line"> System.exit(-<span class="number">3</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> Runtime.getRuntime().addShutdownHook(<span class="keyword">new</span> <span class="title class_">ShutdownHookThread</span>(log, <span class="keyword">new</span> <span class="title class_">Callable</span><Void>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> Void <span class="title function_">call</span><span class="params">()</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> controller.shutdown();</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"> }));</span><br><span class="line"></span><br><span class="line"> controller.start();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> controller;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这个start里主要关注initialize方法，后面就是一个停机的hook，来看下initialize方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">initialize</span><span class="params">()</span> {</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.kvConfigManager.load();</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.remotingServer = <span class="keyword">new</span> <span class="title class_">NettyRemotingServer</span>(<span class="built_in">this</span>.nettyServerConfig, <span class="built_in">this</span>.brokerHousekeepingService);</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.remotingExecutor =</span><br><span class="line"> Executors.newFixedThreadPool(nettyServerConfig.getServerWorkerThreads(), <span class="keyword">new</span> <span class="title class_">ThreadFactoryImpl</span>(<span class="string">"RemotingExecutorThread_"</span>));</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.registerProcessor();</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.scheduledExecutorService.scheduleAtFixedRate(<span class="keyword">new</span> <span class="title class_">Runnable</span>() {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> NamesrvController.<span class="built_in">this</span>.routeInfoManager.scanNotActiveBroker();</span><br><span class="line"> }</span><br><span class="line"> }, <span class="number">5</span>, <span class="number">10</span>, TimeUnit.SECONDS);</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.scheduledExecutorService.scheduleAtFixedRate(<span class="keyword">new</span> <span class="title class_">Runnable</span>() {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> NamesrvController.<span class="built_in">this</span>.kvConfigManager.printAllPeriodically();</span><br><span class="line"> }</span><br><span class="line"> }, <span class="number">1</span>, <span class="number">10</span>, TimeUnit.MINUTES);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (TlsSystemConfig.tlsMode != TlsMode.DISABLED) {</span><br><span class="line"> <span class="comment">// Register a listener to reload SslContext</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> fileWatchService = <span class="keyword">new</span> <span class="title class_">FileWatchService</span>(</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">String</span>[] {</span><br><span class="line"> TlsSystemConfig.tlsServerCertPath,</span><br><span class="line"> TlsSystemConfig.tlsServerKeyPath,</span><br><span class="line"> TlsSystemConfig.tlsServerTrustCertPath</span><br><span class="line"> },</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">FileWatchService</span>.Listener() {</span><br><span class="line"> <span class="type">boolean</span> certChanged, keyChanged = <span class="literal">false</span>;</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">onChanged</span><span class="params">(String path)</span> {</span><br><span class="line"> <span class="keyword">if</span> (path.equals(TlsSystemConfig.tlsServerTrustCertPath)) {</span><br><span class="line"> log.info(<span class="string">"The trust certificate changed, reload the ssl context"</span>);</span><br><span class="line"> reloadServerSslContext();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (path.equals(TlsSystemConfig.tlsServerCertPath)) {</span><br><span class="line"> certChanged = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (path.equals(TlsSystemConfig.tlsServerKeyPath)) {</span><br><span class="line"> keyChanged = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (certChanged && keyChanged) {</span><br><span class="line"> log.info(<span class="string">"The certificate and private key changed, reload the ssl context"</span>);</span><br><span class="line"> certChanged = keyChanged = <span class="literal">false</span>;</span><br><span class="line"> reloadServerSslContext();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">reloadServerSslContext</span><span class="params">()</span> {</span><br><span class="line"> ((NettyRemotingServer) remotingServer).loadSslContext();</span><br><span class="line"> }</span><br><span class="line"> });</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> log.warn(<span class="string">"FileWatchService created error, can't load the certificate dynamically"</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这里的kvConfigManager主要是来加载NameServer的配置参数，存到org.apache.rocketmq.namesrv.kvconfig.KVConfigManager#configTable中，然后是以BrokerHousekeepingService对象为参数初始化NettyRemotingServer对象，BrokerHousekeepingService对象作为该Netty连接中Socket链接的监听器（ChannelEventListener）；监听与Broker建立的渠道的状态（空闲、关闭、异常三个状态），并调用BrokerHousekeepingService的相应onChannel方法。其中渠道的空闲、关闭、异常状态均调用RouteInfoManager.onChannelDestory方法处理。这个BrokerHousekeepingService可以字面化地理解为broker的管家服务，这个类内部三个状态方法其实都是调用的org.apache.rocketmq.namesrv.NamesrvController#getRouteInfoManager方法，而这个RouteInfoManager里面的对象有这些</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">RouteInfoManager</span> {</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">InternalLogger</span> <span class="variable">log</span> <span class="operator">=</span> InternalLoggerFactory.getLogger(LoggerName.NAMESRV_LOGGER_NAME);</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="keyword">static</span> <span class="type">long</span> <span class="variable">BROKER_CHANNEL_EXPIRED_TIME</span> <span class="operator">=</span> <span class="number">1000</span> * <span class="number">60</span> * <span class="number">2</span>;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="type">ReadWriteLock</span> <span class="variable">lock</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ReentrantReadWriteLock</span>();</span><br><span class="line"> <span class="comment">// topic与queue的对应关系</span></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> HashMap<String<span class="comment">/* topic */</span>, List<QueueData>> topicQueueTable;</span><br><span class="line"> <span class="comment">// Broker名称与broker属性的map</span></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> HashMap<String<span class="comment">/* brokerName */</span>, BrokerData> brokerAddrTable;</span><br><span class="line"> <span class="comment">// 集群与broker集合的对应关系</span></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> HashMap<String<span class="comment">/* clusterName */</span>, Set<String<span class="comment">/* brokerName */</span>>> clusterAddrTable;</span><br><span class="line"> <span class="comment">// 活跃的broker信息</span></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> HashMap<String<span class="comment">/* brokerAddr */</span>, BrokerLiveInfo> brokerLiveTable;</span><br><span class="line"> <span class="comment">// Broker地址与过滤器</span></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> HashMap<String<span class="comment">/* brokerAddr */</span>, List<String><span class="comment">/* Filter Server */</span>> filterServerTable;</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后接下去就是初始化了一个线程池，然后注册默认的处理类<code>this.registerProcessor();</code>默认都是这个处理器去处理请求 <code>org.apache.rocketmq.namesrv.processor.DefaultRequestProcessor#DefaultRequestProcessor</code>然后是初始化两个定时任务</p>
|
|
<p>第一是每10秒检查一遍Broker的状态的定时任务，调用scanNotActiveBroker方法；遍历brokerLiveTable集合，查看每个broker的最后更新时间（BrokerLiveInfo.lastUpdateTimestamp）是否超过2分钟，若超过则关闭该broker的渠道并调用RouteInfoManager.onChannelDestory方法清理RouteInfoManager类的topicQueueTable、brokerAddrTable、clusterAddrTable、filterServerTable成员变量。</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="built_in">this</span>.scheduledExecutorService.scheduleAtFixedRate(<span class="keyword">new</span> <span class="title class_">Runnable</span>() {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> NamesrvController.<span class="built_in">this</span>.routeInfoManager.scanNotActiveBroker();</span><br><span class="line"> }</span><br><span class="line"> }, <span class="number">5</span>, <span class="number">10</span>, TimeUnit.SECONDS);</span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">scanNotActiveBroker</span><span class="params">()</span> {</span><br><span class="line"> Iterator<Entry<String, BrokerLiveInfo>> it = <span class="built_in">this</span>.brokerLiveTable.entrySet().iterator();</span><br><span class="line"> <span class="keyword">while</span> (it.hasNext()) {</span><br><span class="line"> Entry<String, BrokerLiveInfo> next = it.next();</span><br><span class="line"> <span class="type">long</span> <span class="variable">last</span> <span class="operator">=</span> next.getValue().getLastUpdateTimestamp();</span><br><span class="line"> <span class="keyword">if</span> ((last + BROKER_CHANNEL_EXPIRED_TIME) < System.currentTimeMillis()) {</span><br><span class="line"> RemotingUtil.closeChannel(next.getValue().getChannel());</span><br><span class="line"> it.remove();</span><br><span class="line"> log.warn(<span class="string">"The broker channel expired, {} {}ms"</span>, next.getKey(), BROKER_CHANNEL_EXPIRED_TIME);</span><br><span class="line"> <span class="built_in">this</span>.onChannelDestroy(next.getKey(), next.getValue().getChannel());</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">onChannelDestroy</span><span class="params">(String remoteAddr, Channel channel)</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">brokerAddrFound</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">if</span> (channel != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="built_in">this</span>.lock.readLock().lockInterruptibly();</span><br><span class="line"> Iterator<Entry<String, BrokerLiveInfo>> itBrokerLiveTable =</span><br><span class="line"> <span class="built_in">this</span>.brokerLiveTable.entrySet().iterator();</span><br><span class="line"> <span class="keyword">while</span> (itBrokerLiveTable.hasNext()) {</span><br><span class="line"> Entry<String, BrokerLiveInfo> entry = itBrokerLiveTable.next();</span><br><span class="line"> <span class="keyword">if</span> (entry.getValue().getChannel() == channel) {</span><br><span class="line"> brokerAddrFound = entry.getKey();</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="built_in">this</span>.lock.readLock().unlock();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> log.error(<span class="string">"onChannelDestroy Exception"</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == brokerAddrFound) {</span><br><span class="line"> brokerAddrFound = remoteAddr;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> log.info(<span class="string">"the broker's channel destroyed, {}, clean it's data structure at once"</span>, brokerAddrFound);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (brokerAddrFound != <span class="literal">null</span> && brokerAddrFound.length() > <span class="number">0</span>) {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="built_in">this</span>.lock.writeLock().lockInterruptibly();</span><br><span class="line"> <span class="built_in">this</span>.brokerLiveTable.remove(brokerAddrFound);</span><br><span class="line"> <span class="built_in">this</span>.filterServerTable.remove(brokerAddrFound);</span><br><span class="line"> <span class="type">String</span> <span class="variable">brokerNameFound</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">removeBrokerName</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> Iterator<Entry<String, BrokerData>> itBrokerAddrTable =</span><br><span class="line"> <span class="built_in">this</span>.brokerAddrTable.entrySet().iterator();</span><br><span class="line"> <span class="keyword">while</span> (itBrokerAddrTable.hasNext() && (<span class="literal">null</span> == brokerNameFound)) {</span><br><span class="line"> <span class="type">BrokerData</span> <span class="variable">brokerData</span> <span class="operator">=</span> itBrokerAddrTable.next().getValue();</span><br><span class="line"></span><br><span class="line"> Iterator<Entry<Long, String>> it = brokerData.getBrokerAddrs().entrySet().iterator();</span><br><span class="line"> <span class="keyword">while</span> (it.hasNext()) {</span><br><span class="line"> Entry<Long, String> entry = it.next();</span><br><span class="line"> <span class="type">Long</span> <span class="variable">brokerId</span> <span class="operator">=</span> entry.getKey();</span><br><span class="line"> <span class="type">String</span> <span class="variable">brokerAddr</span> <span class="operator">=</span> entry.getValue();</span><br><span class="line"> <span class="keyword">if</span> (brokerAddr.equals(brokerAddrFound)) {</span><br><span class="line"> brokerNameFound = brokerData.getBrokerName();</span><br><span class="line"> it.remove();</span><br><span class="line"> log.info(<span class="string">"remove brokerAddr[{}, {}] from brokerAddrTable, because channel destroyed"</span>,</span><br><span class="line"> brokerId, brokerAddr);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (brokerData.getBrokerAddrs().isEmpty()) {</span><br><span class="line"> removeBrokerName = <span class="literal">true</span>;</span><br><span class="line"> itBrokerAddrTable.remove();</span><br><span class="line"> log.info(<span class="string">"remove brokerName[{}] from brokerAddrTable, because channel destroyed"</span>,</span><br><span class="line"> brokerData.getBrokerName());</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (brokerNameFound != <span class="literal">null</span> && removeBrokerName) {</span><br><span class="line"> Iterator<Entry<String, Set<String>>> it = <span class="built_in">this</span>.clusterAddrTable.entrySet().iterator();</span><br><span class="line"> <span class="keyword">while</span> (it.hasNext()) {</span><br><span class="line"> Entry<String, Set<String>> entry = it.next();</span><br><span class="line"> <span class="type">String</span> <span class="variable">clusterName</span> <span class="operator">=</span> entry.getKey();</span><br><span class="line"> Set<String> brokerNames = entry.getValue();</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">removed</span> <span class="operator">=</span> brokerNames.remove(brokerNameFound);</span><br><span class="line"> <span class="keyword">if</span> (removed) {</span><br><span class="line"> log.info(<span class="string">"remove brokerName[{}], clusterName[{}] from clusterAddrTable, because channel destroyed"</span>,</span><br><span class="line"> brokerNameFound, clusterName);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (brokerNames.isEmpty()) {</span><br><span class="line"> log.info(<span class="string">"remove the clusterName[{}] from clusterAddrTable, because channel destroyed and no broker in this cluster"</span>,</span><br><span class="line"> clusterName);</span><br><span class="line"> it.remove();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (removeBrokerName) {</span><br><span class="line"> Iterator<Entry<String, List<QueueData>>> itTopicQueueTable =</span><br><span class="line"> <span class="built_in">this</span>.topicQueueTable.entrySet().iterator();</span><br><span class="line"> <span class="keyword">while</span> (itTopicQueueTable.hasNext()) {</span><br><span class="line"> Entry<String, List<QueueData>> entry = itTopicQueueTable.next();</span><br><span class="line"> <span class="type">String</span> <span class="variable">topic</span> <span class="operator">=</span> entry.getKey();</span><br><span class="line"> List<QueueData> queueDataList = entry.getValue();</span><br><span class="line"></span><br><span class="line"> Iterator<QueueData> itQueueData = queueDataList.iterator();</span><br><span class="line"> <span class="keyword">while</span> (itQueueData.hasNext()) {</span><br><span class="line"> <span class="type">QueueData</span> <span class="variable">queueData</span> <span class="operator">=</span> itQueueData.next();</span><br><span class="line"> <span class="keyword">if</span> (queueData.getBrokerName().equals(brokerNameFound)) {</span><br><span class="line"> itQueueData.remove();</span><br><span class="line"> log.info(<span class="string">"remove topic[{} {}], from topicQueueTable, because channel destroyed"</span>,</span><br><span class="line"> topic, queueData);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (queueDataList.isEmpty()) {</span><br><span class="line"> itTopicQueueTable.remove();</span><br><span class="line"> log.info(<span class="string">"remove topic[{}] all queue, from topicQueueTable, because channel destroyed"</span>,</span><br><span class="line"> topic);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="built_in">this</span>.lock.writeLock().unlock();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> log.error(<span class="string">"onChannelDestroy Exception"</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>第二个是每10分钟打印一次NameServer的配置参数。即KVConfigManager.configTable变量的内容。</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="built_in">this</span>.scheduledExecutorService.scheduleAtFixedRate(<span class="keyword">new</span> <span class="title class_">Runnable</span>() {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> NamesrvController.<span class="built_in">this</span>.kvConfigManager.printAllPeriodically();</span><br><span class="line"> }</span><br><span class="line"> }, <span class="number">1</span>, <span class="number">10</span>, TimeUnit.MINUTES);</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后这个初始化就差不多完成了，后面只需要把remotingServer start一下就好了</p>
|
|
<h3 id="处理请求"><a href="#处理请求" class="headerlink" title="处理请求"></a>处理请求</h3><p>直接上代码，其实主体是swtich case去判断</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> RemotingCommand <span class="title function_">processRequest</span><span class="params">(ChannelHandlerContext ctx,</span></span><br><span class="line"><span class="params"> RemotingCommand request)</span> <span class="keyword">throws</span> RemotingCommandException {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (ctx != <span class="literal">null</span>) {</span><br><span class="line"> log.debug(<span class="string">"receive request, {} {} {}"</span>,</span><br><span class="line"> request.getCode(),</span><br><span class="line"> RemotingHelper.parseChannelRemoteAddr(ctx.channel()),</span><br><span class="line"> request);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="keyword">switch</span> (request.getCode()) {</span><br><span class="line"> <span class="keyword">case</span> RequestCode.PUT_KV_CONFIG:</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.putKVConfig(ctx, request);</span><br><span class="line"> <span class="keyword">case</span> RequestCode.GET_KV_CONFIG:</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.getKVConfig(ctx, request);</span><br><span class="line"> <span class="keyword">case</span> RequestCode.DELETE_KV_CONFIG:</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.deleteKVConfig(ctx, request);</span><br><span class="line"> <span class="keyword">case</span> RequestCode.QUERY_DATA_VERSION:</span><br><span class="line"> <span class="keyword">return</span> queryBrokerTopicConfig(ctx, request);</span><br><span class="line"> <span class="keyword">case</span> RequestCode.REGISTER_BROKER:</span><br><span class="line"> <span class="type">Version</span> <span class="variable">brokerVersion</span> <span class="operator">=</span> MQVersion.value2Version(request.getVersion());</span><br><span class="line"> <span class="keyword">if</span> (brokerVersion.ordinal() >= MQVersion.Version.V3_0_11.ordinal()) {</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.registerBrokerWithFilterServer(ctx, request);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.registerBroker(ctx, request);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">case</span> RequestCode.UNREGISTER_BROKER:</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.unregisterBroker(ctx, request);</span><br><span class="line"> <span class="keyword">case</span> RequestCode.GET_ROUTEINTO_BY_TOPIC:</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.getRouteInfoByTopic(ctx, request);</span><br><span class="line"> <span class="keyword">case</span> RequestCode.GET_BROKER_CLUSTER_INFO:</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.getBrokerClusterInfo(ctx, request);</span><br><span class="line"> <span class="keyword">case</span> RequestCode.WIPE_WRITE_PERM_OF_BROKER:</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.wipeWritePermOfBroker(ctx, request);</span><br><span class="line"> <span class="keyword">case</span> RequestCode.GET_ALL_TOPIC_LIST_FROM_NAMESERVER:</span><br><span class="line"> <span class="keyword">return</span> getAllTopicListFromNameserver(ctx, request);</span><br><span class="line"> <span class="keyword">case</span> RequestCode.DELETE_TOPIC_IN_NAMESRV:</span><br><span class="line"> <span class="keyword">return</span> deleteTopicInNamesrv(ctx, request);</span><br><span class="line"> <span class="keyword">case</span> RequestCode.GET_KVLIST_BY_NAMESPACE:</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.getKVListByNamespace(ctx, request);</span><br><span class="line"> <span class="keyword">case</span> RequestCode.GET_TOPICS_BY_CLUSTER:</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.getTopicsByCluster(ctx, request);</span><br><span class="line"> <span class="keyword">case</span> RequestCode.GET_SYSTEM_TOPIC_LIST_FROM_NS:</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.getSystemTopicListFromNs(ctx, request);</span><br><span class="line"> <span class="keyword">case</span> RequestCode.GET_UNIT_TOPIC_LIST:</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.getUnitTopicList(ctx, request);</span><br><span class="line"> <span class="keyword">case</span> RequestCode.GET_HAS_UNIT_SUB_TOPIC_LIST:</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.getHasUnitSubTopicList(ctx, request);</span><br><span class="line"> <span class="keyword">case</span> RequestCode.GET_HAS_UNIT_SUB_UNUNIT_TOPIC_LIST:</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.getHasUnitSubUnUnitTopicList(ctx, request);</span><br><span class="line"> <span class="keyword">case</span> RequestCode.UPDATE_NAMESRV_CONFIG:</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.updateConfig(ctx, request);</span><br><span class="line"> <span class="keyword">case</span> RequestCode.GET_NAMESRV_CONFIG:</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.getConfig(ctx, request);</span><br><span class="line"> <span class="keyword">default</span>:</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>以broker注册为例，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">case</span> RequestCode.REGISTER_BROKER:</span><br><span class="line"> <span class="type">Version</span> <span class="variable">brokerVersion</span> <span class="operator">=</span> MQVersion.value2Version(request.getVersion());</span><br><span class="line"> <span class="keyword">if</span> (brokerVersion.ordinal() >= MQVersion.Version.V3_0_11.ordinal()) {</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.registerBrokerWithFilterServer(ctx, request);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.registerBroker(ctx, request);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>做了个简单的版本管理，我们看下前面一个的代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> RemotingCommand <span class="title function_">registerBrokerWithFilterServer</span><span class="params">(ChannelHandlerContext ctx, RemotingCommand request)</span></span><br><span class="line"> <span class="keyword">throws</span> RemotingCommandException {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">RemotingCommand</span> <span class="variable">response</span> <span class="operator">=</span> RemotingCommand.createResponseCommand(RegisterBrokerResponseHeader.class);</span><br><span class="line"> <span class="keyword">final</span> <span class="type">RegisterBrokerResponseHeader</span> <span class="variable">responseHeader</span> <span class="operator">=</span> (RegisterBrokerResponseHeader) response.readCustomHeader();</span><br><span class="line"> <span class="keyword">final</span> <span class="type">RegisterBrokerRequestHeader</span> <span class="variable">requestHeader</span> <span class="operator">=</span></span><br><span class="line"> (RegisterBrokerRequestHeader) request.decodeCommandCustomHeader(RegisterBrokerRequestHeader.class);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!checksum(ctx, request, requestHeader)) {</span><br><span class="line"> response.setCode(ResponseCode.SYSTEM_ERROR);</span><br><span class="line"> response.setRemark(<span class="string">"crc32 not match"</span>);</span><br><span class="line"> <span class="keyword">return</span> response;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">RegisterBrokerBody</span> <span class="variable">registerBrokerBody</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">RegisterBrokerBody</span>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (request.getBody() != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> registerBrokerBody = RegisterBrokerBody.decode(request.getBody(), requestHeader.isCompressed());</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">RemotingCommandException</span>(<span class="string">"Failed to decode RegisterBrokerBody"</span>, e);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> registerBrokerBody.getTopicConfigSerializeWrapper().getDataVersion().setCounter(<span class="keyword">new</span> <span class="title class_">AtomicLong</span>(<span class="number">0</span>));</span><br><span class="line"> registerBrokerBody.getTopicConfigSerializeWrapper().getDataVersion().setTimestamp(<span class="number">0</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">RegisterBrokerResult</span> <span class="variable">result</span> <span class="operator">=</span> <span class="built_in">this</span>.namesrvController.getRouteInfoManager().registerBroker(</span><br><span class="line"> requestHeader.getClusterName(),</span><br><span class="line"> requestHeader.getBrokerAddr(),</span><br><span class="line"> requestHeader.getBrokerName(),</span><br><span class="line"> requestHeader.getBrokerId(),</span><br><span class="line"> requestHeader.getHaServerAddr(),</span><br><span class="line"> registerBrokerBody.getTopicConfigSerializeWrapper(),</span><br><span class="line"> registerBrokerBody.getFilterServerList(),</span><br><span class="line"> ctx.channel());</span><br><span class="line"></span><br><span class="line"> responseHeader.setHaServerAddr(result.getHaServerAddr());</span><br><span class="line"> responseHeader.setMasterAddr(result.getMasterAddr());</span><br><span class="line"></span><br><span class="line"> <span class="type">byte</span>[] jsonValue = <span class="built_in">this</span>.namesrvController.getKvConfigManager().getKVListByNamespace(NamesrvUtil.NAMESPACE_ORDER_TOPIC_CONFIG);</span><br><span class="line"> response.setBody(jsonValue);</span><br><span class="line"></span><br><span class="line"> response.setCode(ResponseCode.SUCCESS);</span><br><span class="line"> response.setRemark(<span class="literal">null</span>);</span><br><span class="line"> <span class="keyword">return</span> response;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>可以看到主要的逻辑还是在<code>org.apache.rocketmq.namesrv.routeinfo.RouteInfoManager#registerBroker</code>这个方法里</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> RegisterBrokerResult <span class="title function_">registerBroker</span><span class="params">(</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> String clusterName,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> String brokerAddr,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> String brokerName,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> <span class="type">long</span> brokerId,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> String haServerAddr,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> TopicConfigSerializeWrapper topicConfigWrapper,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> List<String> filterServerList,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> Channel channel)</span> {</span><br><span class="line"> <span class="type">RegisterBrokerResult</span> <span class="variable">result</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">RegisterBrokerResult</span>();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="built_in">this</span>.lock.writeLock().lockInterruptibly();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 更新这个clusterAddrTable</span></span><br><span class="line"> Set<String> brokerNames = <span class="built_in">this</span>.clusterAddrTable.get(clusterName);</span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == brokerNames) {</span><br><span class="line"> brokerNames = <span class="keyword">new</span> <span class="title class_">HashSet</span><String>();</span><br><span class="line"> <span class="built_in">this</span>.clusterAddrTable.put(clusterName, brokerNames);</span><br><span class="line"> }</span><br><span class="line"> brokerNames.add(brokerName);</span><br><span class="line"></span><br><span class="line"> <span class="type">boolean</span> <span class="variable">registerFirst</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 更新brokerAddrTable</span></span><br><span class="line"> <span class="type">BrokerData</span> <span class="variable">brokerData</span> <span class="operator">=</span> <span class="built_in">this</span>.brokerAddrTable.get(brokerName);</span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == brokerData) {</span><br><span class="line"> registerFirst = <span class="literal">true</span>;</span><br><span class="line"> brokerData = <span class="keyword">new</span> <span class="title class_">BrokerData</span>(clusterName, brokerName, <span class="keyword">new</span> <span class="title class_">HashMap</span><Long, String>());</span><br><span class="line"> <span class="built_in">this</span>.brokerAddrTable.put(brokerName, brokerData);</span><br><span class="line"> }</span><br><span class="line"> Map<Long, String> brokerAddrsMap = brokerData.getBrokerAddrs();</span><br><span class="line"> <span class="comment">//Switch slave to master: first remove <1, IP:PORT> in namesrv, then add <0, IP:PORT></span></span><br><span class="line"> <span class="comment">//The same IP:PORT must only have one record in brokerAddrTable</span></span><br><span class="line"> Iterator<Entry<Long, String>> it = brokerAddrsMap.entrySet().iterator();</span><br><span class="line"> <span class="keyword">while</span> (it.hasNext()) {</span><br><span class="line"> Entry<Long, String> item = it.next();</span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> != brokerAddr && brokerAddr.equals(item.getValue()) && brokerId != item.getKey()) {</span><br><span class="line"> it.remove();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">String</span> <span class="variable">oldAddr</span> <span class="operator">=</span> brokerData.getBrokerAddrs().put(brokerId, brokerAddr);</span><br><span class="line"> registerFirst = registerFirst || (<span class="literal">null</span> == oldAddr);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 更新了org.apache.rocketmq.namesrv.routeinfo.RouteInfoManager#topicQueueTable中的数据</span></span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> != topicConfigWrapper</span><br><span class="line"> && MixAll.MASTER_ID == brokerId) {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.isBrokerTopicConfigChanged(brokerAddr, topicConfigWrapper.getDataVersion())</span><br><span class="line"> || registerFirst) {</span><br><span class="line"> ConcurrentMap<String, TopicConfig> tcTable =</span><br><span class="line"> topicConfigWrapper.getTopicConfigTable();</span><br><span class="line"> <span class="keyword">if</span> (tcTable != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">for</span> (Map.Entry<String, TopicConfig> entry : tcTable.entrySet()) {</span><br><span class="line"> <span class="built_in">this</span>.createAndUpdateQueueData(brokerName, entry.getValue());</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 更新活跃broker信息</span></span><br><span class="line"> <span class="type">BrokerLiveInfo</span> <span class="variable">prevBrokerLiveInfo</span> <span class="operator">=</span> <span class="built_in">this</span>.brokerLiveTable.put(brokerAddr,</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">BrokerLiveInfo</span>(</span><br><span class="line"> System.currentTimeMillis(),</span><br><span class="line"> topicConfigWrapper.getDataVersion(),</span><br><span class="line"> channel,</span><br><span class="line"> haServerAddr));</span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == prevBrokerLiveInfo) {</span><br><span class="line"> log.info(<span class="string">"new broker registered, {} HAServer: {}"</span>, brokerAddr, haServerAddr);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 处理filter</span></span><br><span class="line"> <span class="keyword">if</span> (filterServerList != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">if</span> (filterServerList.isEmpty()) {</span><br><span class="line"> <span class="built_in">this</span>.filterServerTable.remove(brokerAddr);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="built_in">this</span>.filterServerTable.put(brokerAddr, filterServerList);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 当当前broker非master时返回master信息</span></span><br><span class="line"> <span class="keyword">if</span> (MixAll.MASTER_ID != brokerId) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">masterAddr</span> <span class="operator">=</span> brokerData.getBrokerAddrs().get(MixAll.MASTER_ID);</span><br><span class="line"> <span class="keyword">if</span> (masterAddr != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">BrokerLiveInfo</span> <span class="variable">brokerLiveInfo</span> <span class="operator">=</span> <span class="built_in">this</span>.brokerLiveTable.get(masterAddr);</span><br><span class="line"> <span class="keyword">if</span> (brokerLiveInfo != <span class="literal">null</span>) {</span><br><span class="line"> result.setHaServerAddr(brokerLiveInfo.getHaServerAddr());</span><br><span class="line"> result.setMasterAddr(masterAddr);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="built_in">this</span>.lock.writeLock().unlock();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> log.error(<span class="string">"registerBroker Exception"</span>, e);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> result;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这个是注册 broker 的逻辑，再看下根据 topic 获取 broker 信息和 topic 信息，<code>org.apache.rocketmq.namesrv.processor.DefaultRequestProcessor#getRouteInfoByTopic</code> 主要是这个方法的逻辑</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> RemotingCommand <span class="title function_">getRouteInfoByTopic</span><span class="params">(ChannelHandlerContext ctx,</span></span><br><span class="line"><span class="params"> RemotingCommand request)</span> <span class="keyword">throws</span> RemotingCommandException {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">RemotingCommand</span> <span class="variable">response</span> <span class="operator">=</span> RemotingCommand.createResponseCommand(<span class="literal">null</span>);</span><br><span class="line"> <span class="keyword">final</span> <span class="type">GetRouteInfoRequestHeader</span> <span class="variable">requestHeader</span> <span class="operator">=</span></span><br><span class="line"> (GetRouteInfoRequestHeader) request.decodeCommandCustomHeader(GetRouteInfoRequestHeader.class);</span><br><span class="line"></span><br><span class="line"> <span class="type">TopicRouteData</span> <span class="variable">topicRouteData</span> <span class="operator">=</span> <span class="built_in">this</span>.namesrvController.getRouteInfoManager().pickupTopicRouteData(requestHeader.getTopic());</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (topicRouteData != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.namesrvController.getNamesrvConfig().isOrderMessageEnable()) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">orderTopicConf</span> <span class="operator">=</span></span><br><span class="line"> <span class="built_in">this</span>.namesrvController.getKvConfigManager().getKVConfig(NamesrvUtil.NAMESPACE_ORDER_TOPIC_CONFIG,</span><br><span class="line"> requestHeader.getTopic());</span><br><span class="line"> topicRouteData.setOrderTopicConf(orderTopicConf);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">byte</span>[] content = topicRouteData.encode();</span><br><span class="line"> response.setBody(content);</span><br><span class="line"> response.setCode(ResponseCode.SUCCESS);</span><br><span class="line"> response.setRemark(<span class="literal">null</span>);</span><br><span class="line"> <span class="keyword">return</span> response;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> response.setCode(ResponseCode.TOPIC_NOT_EXIST);</span><br><span class="line"> response.setRemark(<span class="string">"No topic route info in name server for the topic: "</span> + requestHeader.getTopic()</span><br><span class="line"> + FAQUrl.suggestTodo(FAQUrl.APPLY_TOPIC_URL));</span><br><span class="line"> <span class="keyword">return</span> response;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>首先调用<code>org.apache.rocketmq.namesrv.routeinfo.RouteInfoManager#pickupTopicRouteData</code>从<code>org.apache.rocketmq.namesrv.routeinfo.RouteInfoManager#topicQueueTable</code>获取到<code>org.apache.rocketmq.common.protocol.route.QueueData</code>这里面存了 brokerName，再通过<code>org.apache.rocketmq.namesrv.routeinfo.RouteInfoManager#brokerAddrTable</code>里获取到 broker 的地址信息等，然后再获取 orderMessage 的配置。</p>
|
|
<p>简要分析了下 RocketMQ 的 NameServer 的代码，比较粗粒度。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>MQ</category>
|
|
<category>RocketMQ</category>
|
|
<category>消息队列</category>
|
|
<category>RocketMQ</category>
|
|
<category>中间件</category>
|
|
<category>RocketMQ</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>MQ</tag>
|
|
<tag>消息队列</tag>
|
|
<tag>RocketMQ</tag>
|
|
<tag>削峰填谷</tag>
|
|
<tag>中间件</tag>
|
|
<tag>源码解析</tag>
|
|
<tag>NameServer</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊一下 RocketMQ 的消息存储三</title>
|
|
<url>/2021/10/03/%E8%81%8A%E4%B8%80%E4%B8%8B-RocketMQ-%E7%9A%84%E6%B6%88%E6%81%AF%E5%AD%98%E5%82%A8%E4%B8%89/</url>
|
|
<content><![CDATA[<p>ConsumeQueue 其实是定位到一个 topic 下的消息在 CommitLog 下的偏移量，它也是固定大小的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// ConsumeQueue file size,default is 30W</span></span><br><span class="line"><span class="keyword">private</span> <span class="type">int</span> <span class="variable">mapedFileSizeConsumeQueue</span> <span class="operator">=</span> <span class="number">300000</span> * ConsumeQueue.CQ_STORE_UNIT_SIZE;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">CQ_STORE_UNIT_SIZE</span> <span class="operator">=</span> <span class="number">20</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>所以文件大小是5.7M 左右</p>
|
|
<p><img data-src="https://img.nicksxs.me/uPic/5udpag.png" alt="5udpag"></p>
|
|
<p>ConsumeQueue 的构建是通过<code>org.apache.rocketmq.store.DefaultMessageStore.ReputMessageService</code>运行后的 doReput 方法，而启动是的 reputFromOffset 则是通过<code>org.apache.rocketmq.store.DefaultMessageStore#start</code>中下面代码设置并启动</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">log.info(<span class="string">"[SetReputOffset] maxPhysicalPosInLogicQueue={} clMinOffset={} clMaxOffset={} clConfirmedOffset={}"</span>,</span><br><span class="line"> maxPhysicalPosInLogicQueue, <span class="built_in">this</span>.commitLog.getMinOffset(), <span class="built_in">this</span>.commitLog.getMaxOffset(), <span class="built_in">this</span>.commitLog.getConfirmOffset());</span><br><span class="line"> <span class="built_in">this</span>.reputMessageService.setReputFromOffset(maxPhysicalPosInLogicQueue);</span><br><span class="line"> <span class="built_in">this</span>.reputMessageService.start();</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>看一下 doReput 的逻辑</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">doReput</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.reputFromOffset < DefaultMessageStore.<span class="built_in">this</span>.commitLog.getMinOffset()) {</span><br><span class="line"> log.warn(<span class="string">"The reputFromOffset={} is smaller than minPyOffset={}, this usually indicate that the dispatch behind too much and the commitlog has expired."</span>,</span><br><span class="line"> <span class="built_in">this</span>.reputFromOffset, DefaultMessageStore.<span class="built_in">this</span>.commitLog.getMinOffset());</span><br><span class="line"> <span class="built_in">this</span>.reputFromOffset = DefaultMessageStore.<span class="built_in">this</span>.commitLog.getMinOffset();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">boolean</span> <span class="variable">doNext</span> <span class="operator">=</span> <span class="literal">true</span>; <span class="built_in">this</span>.isCommitLogAvailable() && doNext; ) {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (DefaultMessageStore.<span class="built_in">this</span>.getMessageStoreConfig().isDuplicationEnable()</span><br><span class="line"> && <span class="built_in">this</span>.reputFromOffset >= DefaultMessageStore.<span class="built_in">this</span>.getConfirmOffset()) {</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 根据偏移量获取消息</span></span><br><span class="line"> <span class="type">SelectMappedBufferResult</span> <span class="variable">result</span> <span class="operator">=</span> DefaultMessageStore.<span class="built_in">this</span>.commitLog.getData(reputFromOffset);</span><br><span class="line"> <span class="keyword">if</span> (result != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="built_in">this</span>.reputFromOffset = result.getStartOffset();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">readSize</span> <span class="operator">=</span> <span class="number">0</span>; readSize < result.getSize() && doNext; ) {</span><br><span class="line"> <span class="comment">// 消息校验和转换</span></span><br><span class="line"> <span class="type">DispatchRequest</span> <span class="variable">dispatchRequest</span> <span class="operator">=</span></span><br><span class="line"> DefaultMessageStore.<span class="built_in">this</span>.commitLog.checkMessageAndReturnSize(result.getByteBuffer(), <span class="literal">false</span>, <span class="literal">false</span>);</span><br><span class="line"> <span class="type">int</span> <span class="variable">size</span> <span class="operator">=</span> dispatchRequest.getBufferSize() == -<span class="number">1</span> ? dispatchRequest.getMsgSize() : dispatchRequest.getBufferSize();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (dispatchRequest.isSuccess()) {</span><br><span class="line"> <span class="keyword">if</span> (size > <span class="number">0</span>) {</span><br><span class="line"> <span class="comment">// 进行分发处理，包括 ConsumeQueue 和 IndexFile</span></span><br><span class="line"> DefaultMessageStore.<span class="built_in">this</span>.doDispatch(dispatchRequest);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (BrokerRole.SLAVE != DefaultMessageStore.<span class="built_in">this</span>.getMessageStoreConfig().getBrokerRole()</span><br><span class="line"> && DefaultMessageStore.<span class="built_in">this</span>.brokerConfig.isLongPollingEnable()) {</span><br><span class="line"> DefaultMessageStore.<span class="built_in">this</span>.messageArrivingListener.arriving(dispatchRequest.getTopic(),</span><br><span class="line"> dispatchRequest.getQueueId(), dispatchRequest.getConsumeQueueOffset() + <span class="number">1</span>,</span><br><span class="line"> dispatchRequest.getTagsCode(), dispatchRequest.getStoreTimestamp(),</span><br><span class="line"> dispatchRequest.getBitMap(), dispatchRequest.getPropertiesMap());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.reputFromOffset += size;</span><br><span class="line"> readSize += size;</span><br><span class="line"> <span class="keyword">if</span> (DefaultMessageStore.<span class="built_in">this</span>.getMessageStoreConfig().getBrokerRole() == BrokerRole.SLAVE) {</span><br><span class="line"> DefaultMessageStore.<span class="built_in">this</span>.storeStatsService</span><br><span class="line"> .getSinglePutMessageTopicTimesTotal(dispatchRequest.getTopic()).incrementAndGet();</span><br><span class="line"> DefaultMessageStore.<span class="built_in">this</span>.storeStatsService</span><br><span class="line"> .getSinglePutMessageTopicSizeTotal(dispatchRequest.getTopic())</span><br><span class="line"> .addAndGet(dispatchRequest.getMsgSize());</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (size == <span class="number">0</span>) {</span><br><span class="line"> <span class="built_in">this</span>.reputFromOffset = DefaultMessageStore.<span class="built_in">this</span>.commitLog.rollNextFile(<span class="built_in">this</span>.reputFromOffset);</span><br><span class="line"> readSize = result.getSize();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (!dispatchRequest.isSuccess()) {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (size > <span class="number">0</span>) {</span><br><span class="line"> log.error(<span class="string">"[BUG]read total count not equals msg total size. reputFromOffset={}"</span>, reputFromOffset);</span><br><span class="line"> <span class="built_in">this</span>.reputFromOffset += size;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> doNext = <span class="literal">false</span>;</span><br><span class="line"> <span class="comment">// If user open the dledger pattern or the broker is master node,</span></span><br><span class="line"> <span class="comment">// it will not ignore the exception and fix the reputFromOffset variable</span></span><br><span class="line"> <span class="keyword">if</span> (DefaultMessageStore.<span class="built_in">this</span>.getMessageStoreConfig().isEnableDLegerCommitLog() ||</span><br><span class="line"> DefaultMessageStore.<span class="built_in">this</span>.brokerConfig.getBrokerId() == MixAll.MASTER_ID) {</span><br><span class="line"> log.error(<span class="string">"[BUG]dispatch message to consume queue error, COMMITLOG OFFSET: {}"</span>,</span><br><span class="line"> <span class="built_in">this</span>.reputFromOffset);</span><br><span class="line"> <span class="built_in">this</span>.reputFromOffset += result.getSize() - readSize;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> result.release();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> doNext = <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>分发的逻辑看到这</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"> <span class="keyword">class</span> <span class="title class_">CommitLogDispatcherBuildConsumeQueue</span> <span class="keyword">implements</span> <span class="title class_">CommitLogDispatcher</span> {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">dispatch</span><span class="params">(DispatchRequest request)</span> {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">tranType</span> <span class="operator">=</span> MessageSysFlag.getTransactionValue(request.getSysFlag());</span><br><span class="line"> <span class="keyword">switch</span> (tranType) {</span><br><span class="line"> <span class="keyword">case</span> MessageSysFlag.TRANSACTION_NOT_TYPE:</span><br><span class="line"> <span class="keyword">case</span> MessageSysFlag.TRANSACTION_COMMIT_TYPE:</span><br><span class="line"> DefaultMessageStore.<span class="built_in">this</span>.putMessagePositionInfo(request);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">case</span> MessageSysFlag.TRANSACTION_PREPARED_TYPE:</span><br><span class="line"> <span class="keyword">case</span> MessageSysFlag.TRANSACTION_ROLLBACK_TYPE:</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">putMessagePositionInfo</span><span class="params">(DispatchRequest dispatchRequest)</span> {</span><br><span class="line"> <span class="type">ConsumeQueue</span> <span class="variable">cq</span> <span class="operator">=</span> <span class="built_in">this</span>.findConsumeQueue(dispatchRequest.getTopic(), dispatchRequest.getQueueId());</span><br><span class="line"> cq.putMessagePositionInfoWrapper(dispatchRequest);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>真正存储的是在这</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="type">boolean</span> <span class="title function_">putMessagePositionInfo</span><span class="params">(<span class="keyword">final</span> <span class="type">long</span> offset, <span class="keyword">final</span> <span class="type">int</span> size, <span class="keyword">final</span> <span class="type">long</span> tagsCode,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> <span class="type">long</span> cqOffset)</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (offset + size <= <span class="built_in">this</span>.maxPhysicOffset) {</span><br><span class="line"> log.warn(<span class="string">"Maybe try to build consume queue repeatedly maxPhysicOffset={} phyOffset={}"</span>, maxPhysicOffset, offset);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.byteBufferIndex.flip();</span><br><span class="line"> <span class="built_in">this</span>.byteBufferIndex.limit(CQ_STORE_UNIT_SIZE);</span><br><span class="line"> <span class="built_in">this</span>.byteBufferIndex.putLong(offset);</span><br><span class="line"> <span class="built_in">this</span>.byteBufferIndex.putInt(size);</span><br><span class="line"> <span class="built_in">this</span>.byteBufferIndex.putLong(tagsCode);</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这里也可以看到 ConsumeQueue 的存储格式，</p>
|
|
<p><img data-src="https://img.nicksxs.me/uPic/AA6Tve.jpg" alt="AA6Tve"></p>
|
|
<p>偏移量，消息大小，跟 tag 的 hashCode</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>MQ</category>
|
|
<category>RocketMQ</category>
|
|
<category>消息队列</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>MQ</tag>
|
|
<tag>消息队列</tag>
|
|
<tag>RocketMQ</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>结合本地部署的蒸馏 deepseek 大模型和 Anything LLM 来实现rag功能</title>
|
|
<url>/2025/03/02/%E7%BB%93%E5%90%88%E6%9C%AC%E5%9C%B0%E9%83%A8%E7%BD%B2%E7%9A%84%E8%92%B8%E9%A6%8Fdeepseek%E6%9D%A5%E5%AE%9E%E7%8E%B0rag%E5%8A%9F%E8%83%BD/</url>
|
|
<content><![CDATA[<p>之前我们用LM Studio 本地部署了 deepseek的蒸馏大模型，虽然肯定无法跟满血版比，但是对于本地的一些小应用还是可以尝试的<br>这边我们就不自己实现了，借助于 Anything LLM来做个尝试<br>首先可以在 <a href="https://anythingllm.com/desktop">anythingllm</a> 下载对应系统版本的 Anything，注意安装的时候它有两个阶段，会走两次进度条，第二次会慢一些。<br>安装好后，我们要选择它的大模型供应者<br><img data-src="https://img.nicksxs.me/blog/%E5%BE%AE%E4%BF%A1%E6%88%AA%E5%9B%BE_20250302215705.png"><br>先要在LM Studio中开启http服务，<br>然后再在 AnythingLLM 中选好以 LM Studio 作为大模型提供商<br><img data-src="https://img.nicksxs.me/blog/%E5%BE%AE%E4%BF%A1%E5%9B%BE%E7%89%87_20250302215641.png"><br>之后在 Anything 窗口中可以新建一个工作区<br><img data-src="https://img.nicksxs.me/blog/gGXjjO.jpg"><br>新建完之后，我们为了实现rag功能，就用个简单的示例，比如以raid 作为话题，我找到某乎的一篇文章<br>点击左侧工作区旁边的上传<br><img data-src="https://img.nicksxs.me/blog/%E5%BE%AE%E4%BF%A1%E6%88%AA%E5%9B%BE_20250302220649.png"><br>我们可以这里提交链接，由Anything LLM帮我们把网页保存下来然后embedding到向量数据库里，然后就可以在我们对话的时候根据这个文章来做增项<br><img data-src="https://img.nicksxs.me/blog/%E5%BE%AE%E4%BF%A1%E6%88%AA%E5%9B%BE_20250302220736.png"><br>然后我们问个问题，就可以发现它已经借助于本地文档的信息来增强内容生成<br><img data-src="https://img.nicksxs.me/blog/%E5%BE%AE%E4%BF%A1%E6%88%AA%E5%9B%BE_20250302220933.png"><br>这样就非常方便的实现了rag，还可以提交本地的笔记文稿等，现在这些应用也已经变得越来越成熟了。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊一下 RocketMQ 的消息存储二</title>
|
|
<url>/2021/09/12/%E8%81%8A%E4%B8%80%E4%B8%8B-RocketMQ-%E7%9A%84%E6%B6%88%E6%81%AF%E5%AD%98%E5%82%A8%E4%BA%8C/</url>
|
|
<content><![CDATA[<h3 id="CommitLog-结构"><a href="#CommitLog-结构" class="headerlink" title="CommitLog 结构"></a>CommitLog 结构</h3><p>CommitLog 是 rocketmq 的服务端，也就是 broker 存储消息的的文件，跟 kafka 一样，也是顺序写入，当然消息是变长的，生成的规则是每个文件的默认1G =1024 * 1024 * 1024，commitlog的文件名fileName，名字长度为20位，左边补零，剩余为起始偏移量；比如00000000000000000000代表了第一个文件，起始偏移量为0，文件大小为1G=1 073 741 824Byte；当这个文件满了，第二个文件名字为00000000001073741824，起始偏移量为1073741824, 消息存储的时候会顺序写入文件，当文件满了则写入下一个文件，代码中的定义</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// CommitLog file size,default is 1G</span></span><br><span class="line"><span class="keyword">private</span> <span class="type">int</span> <span class="variable">mapedFileSizeCommitLog</span> <span class="operator">=</span> <span class="number">1024</span> * <span class="number">1024</span> * <span class="number">1024</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<p><img data-src="https://img.nicksxs.me/uPic/kLahwW.png" alt="kLahwW"></p>
|
|
<p>本地跑个 demo 验证下，也是这样，这里奇妙有几个比较巧妙的点（个人观点），首先文件就刚好是 1G，并且按照大小偏移量去生成下一个文件，这样获取消息的时候按大小算一下就知道在哪个文件里了，</p>
|
|
<p>代码中写入 CommitLog 的逻辑可以从这开始看</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> PutMessageResult <span class="title function_">putMessage</span><span class="params">(<span class="keyword">final</span> MessageExtBrokerInner msg)</span> {</span><br><span class="line"> <span class="comment">// Set the storage time</span></span><br><span class="line"> msg.setStoreTimestamp(System.currentTimeMillis());</span><br><span class="line"> <span class="comment">// Set the message body BODY CRC (consider the most appropriate setting</span></span><br><span class="line"> <span class="comment">// on the client)</span></span><br><span class="line"> msg.setBodyCRC(UtilAll.crc32(msg.getBody()));</span><br><span class="line"> <span class="comment">// Back to Results</span></span><br><span class="line"> <span class="type">AppendMessageResult</span> <span class="variable">result</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"></span><br><span class="line"> <span class="type">StoreStatsService</span> <span class="variable">storeStatsService</span> <span class="operator">=</span> <span class="built_in">this</span>.defaultMessageStore.getStoreStatsService();</span><br><span class="line"></span><br><span class="line"> <span class="type">String</span> <span class="variable">topic</span> <span class="operator">=</span> msg.getTopic();</span><br><span class="line"> <span class="type">int</span> <span class="variable">queueId</span> <span class="operator">=</span> msg.getQueueId();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">tranType</span> <span class="operator">=</span> MessageSysFlag.getTransactionValue(msg.getSysFlag());</span><br><span class="line"> <span class="keyword">if</span> (tranType == MessageSysFlag.TRANSACTION_NOT_TYPE</span><br><span class="line"> || tranType == MessageSysFlag.TRANSACTION_COMMIT_TYPE) {</span><br><span class="line"> <span class="comment">// Delay Delivery</span></span><br><span class="line"> <span class="keyword">if</span> (msg.getDelayTimeLevel() > <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">if</span> (msg.getDelayTimeLevel() > <span class="built_in">this</span>.defaultMessageStore.getScheduleMessageService().getMaxDelayLevel()) {</span><br><span class="line"> msg.setDelayTimeLevel(<span class="built_in">this</span>.defaultMessageStore.getScheduleMessageService().getMaxDelayLevel());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> topic = ScheduleMessageService.SCHEDULE_TOPIC;</span><br><span class="line"> queueId = ScheduleMessageService.delayLevel2QueueId(msg.getDelayTimeLevel());</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Backup real topic, queueId</span></span><br><span class="line"> MessageAccessor.putProperty(msg, MessageConst.PROPERTY_REAL_TOPIC, msg.getTopic());</span><br><span class="line"> MessageAccessor.putProperty(msg, MessageConst.PROPERTY_REAL_QUEUE_ID, String.valueOf(msg.getQueueId()));</span><br><span class="line"> msg.setPropertiesString(MessageDecoder.messageProperties2String(msg.getProperties()));</span><br><span class="line"></span><br><span class="line"> msg.setTopic(topic);</span><br><span class="line"> msg.setQueueId(queueId);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">long</span> <span class="variable">eclipseTimeInLock</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="type">MappedFile</span> <span class="variable">unlockMappedFile</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="type">MappedFile</span> <span class="variable">mappedFile</span> <span class="operator">=</span> <span class="built_in">this</span>.mappedFileQueue.getLastMappedFile();</span><br><span class="line"></span><br><span class="line"> putMessageLock.lock(); <span class="comment">//spin or ReentrantLock ,depending on store config</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">long</span> <span class="variable">beginLockTimestamp</span> <span class="operator">=</span> <span class="built_in">this</span>.defaultMessageStore.getSystemClock().now();</span><br><span class="line"> <span class="built_in">this</span>.beginTimeInLock = beginLockTimestamp;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Here settings are stored timestamp, in order to ensure an orderly</span></span><br><span class="line"> <span class="comment">// global</span></span><br><span class="line"> msg.setStoreTimestamp(beginLockTimestamp);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == mappedFile || mappedFile.isFull()) {</span><br><span class="line"> mappedFile = <span class="built_in">this</span>.mappedFileQueue.getLastMappedFile(<span class="number">0</span>); <span class="comment">// Mark: NewFile may be cause noise</span></span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == mappedFile) {</span><br><span class="line"> log.error(<span class="string">"create mapped file1 error, topic: "</span> + msg.getTopic() + <span class="string">" clientAddr: "</span> + msg.getBornHostString());</span><br><span class="line"> beginTimeInLock = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">PutMessageResult</span>(PutMessageStatus.CREATE_MAPEDFILE_FAILED, <span class="literal">null</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> result = mappedFile.appendMessage(msg, <span class="built_in">this</span>.appendMessageCallback);</span><br><span class="line"> <span class="keyword">switch</span> (result.getStatus()) {</span><br><span class="line"> <span class="keyword">case</span> PUT_OK:</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">case</span> END_OF_FILE:</span><br><span class="line"> unlockMappedFile = mappedFile;</span><br><span class="line"> <span class="comment">// Create a new file, re-write the message</span></span><br><span class="line"> mappedFile = <span class="built_in">this</span>.mappedFileQueue.getLastMappedFile(<span class="number">0</span>);</span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == mappedFile) {</span><br><span class="line"> <span class="comment">// <span class="doctag">XXX:</span> warn and notify me</span></span><br><span class="line"> log.error(<span class="string">"create mapped file2 error, topic: "</span> + msg.getTopic() + <span class="string">" clientAddr: "</span> + msg.getBornHostString());</span><br><span class="line"> beginTimeInLock = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">PutMessageResult</span>(PutMessageStatus.CREATE_MAPEDFILE_FAILED, result);</span><br><span class="line"> }</span><br><span class="line"> result = mappedFile.appendMessage(msg, <span class="built_in">this</span>.appendMessageCallback);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">case</span> MESSAGE_SIZE_EXCEEDED:</span><br><span class="line"> <span class="keyword">case</span> PROPERTIES_SIZE_EXCEEDED:</span><br><span class="line"> beginTimeInLock = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">PutMessageResult</span>(PutMessageStatus.MESSAGE_ILLEGAL, result);</span><br><span class="line"> <span class="keyword">case</span> UNKNOWN_ERROR:</span><br><span class="line"> beginTimeInLock = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">PutMessageResult</span>(PutMessageStatus.UNKNOWN_ERROR, result);</span><br><span class="line"> <span class="keyword">default</span>:</span><br><span class="line"> beginTimeInLock = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">PutMessageResult</span>(PutMessageStatus.UNKNOWN_ERROR, result);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> eclipseTimeInLock = <span class="built_in">this</span>.defaultMessageStore.getSystemClock().now() - beginLockTimestamp;</span><br><span class="line"> beginTimeInLock = <span class="number">0</span>;</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> putMessageLock.unlock();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (eclipseTimeInLock > <span class="number">500</span>) {</span><br><span class="line"> log.warn(<span class="string">"[NOTIFYME]putMessage in lock cost time(ms)={}, bodyLength={} AppendMessageResult={}"</span>, eclipseTimeInLock, msg.getBody().length, result);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> != unlockMappedFile && <span class="built_in">this</span>.defaultMessageStore.getMessageStoreConfig().isWarmMapedFileEnable()) {</span><br><span class="line"> <span class="built_in">this</span>.defaultMessageStore.unlockMappedFile(unlockMappedFile);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">PutMessageResult</span> <span class="variable">putMessageResult</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">PutMessageResult</span>(PutMessageStatus.PUT_OK, result);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Statistics</span></span><br><span class="line"> storeStatsService.getSinglePutMessageTopicTimesTotal(msg.getTopic()).incrementAndGet();</span><br><span class="line"> storeStatsService.getSinglePutMessageTopicSizeTotal(topic).addAndGet(result.getWroteBytes());</span><br><span class="line"></span><br><span class="line"> handleDiskFlush(result, putMessageResult, msg);</span><br><span class="line"> handleHA(result, putMessageResult, msg);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> putMessageResult;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>前面也看到在CommitLog 目录下是有大小为 1G 的文件组成，在实现逻辑中，其实是通过 <code>org.apache.rocketmq.store.MappedFileQueue</code> ，内部是存的一个<code>MappedFile</code>的队列，对于写入的场景每次都是通过<code>org.apache.rocketmq.store.MappedFileQueue#getLastMappedFile()</code> 获取最后一个文件，如果还没有创建，或者最后这个文件已经满了，那就调用 <code>org.apache.rocketmq.store.MappedFileQueue#getLastMappedFile(long)</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> MappedFile <span class="title function_">getLastMappedFile</span><span class="params">(<span class="keyword">final</span> <span class="type">long</span> startOffset, <span class="type">boolean</span> needCreate)</span> {</span><br><span class="line"> <span class="type">long</span> <span class="variable">createOffset</span> <span class="operator">=</span> -<span class="number">1</span>;</span><br><span class="line"> <span class="comment">// 调用前面的方法，只是从 mappedFileQueue 获取最后一个</span></span><br><span class="line"> <span class="type">MappedFile</span> <span class="variable">mappedFileLast</span> <span class="operator">=</span> getLastMappedFile();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 如果为空，计算下创建的偏移量</span></span><br><span class="line"> <span class="keyword">if</span> (mappedFileLast == <span class="literal">null</span>) {</span><br><span class="line"> createOffset = startOffset - (startOffset % <span class="built_in">this</span>.mappedFileSize);</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="comment">// 如果不为空，但是当前的文件写满了</span></span><br><span class="line"> <span class="keyword">if</span> (mappedFileLast != <span class="literal">null</span> && mappedFileLast.isFull()) {</span><br><span class="line"> <span class="comment">// 前一个的偏移量加上单个文件的偏移量，也就是 1G</span></span><br><span class="line"> createOffset = mappedFileLast.getFileFromOffset() + <span class="built_in">this</span>.mappedFileSize;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (createOffset != -<span class="number">1</span> && needCreate) {</span><br><span class="line"> <span class="comment">// 根据 createOffset 转换成文件名进行创建</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">nextFilePath</span> <span class="operator">=</span> <span class="built_in">this</span>.storePath + File.separator + UtilAll.offset2FileName(createOffset);</span><br><span class="line"> <span class="type">String</span> <span class="variable">nextNextFilePath</span> <span class="operator">=</span> <span class="built_in">this</span>.storePath + File.separator</span><br><span class="line"> + UtilAll.offset2FileName(createOffset + <span class="built_in">this</span>.mappedFileSize);</span><br><span class="line"> <span class="type">MappedFile</span> <span class="variable">mappedFile</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 这里如果allocateMappedFileService 存在，就提交请求</span></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.allocateMappedFileService != <span class="literal">null</span>) {</span><br><span class="line"> mappedFile = <span class="built_in">this</span>.allocateMappedFileService.putRequestAndReturnMappedFile(nextFilePath,</span><br><span class="line"> nextNextFilePath, <span class="built_in">this</span>.mappedFileSize);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 否则就直接创建</span></span><br><span class="line"> mappedFile = <span class="keyword">new</span> <span class="title class_">MappedFile</span>(nextFilePath, <span class="built_in">this</span>.mappedFileSize);</span><br><span class="line"> } <span class="keyword">catch</span> (IOException e) {</span><br><span class="line"> log.error(<span class="string">"create mappedFile exception"</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (mappedFile != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.mappedFiles.isEmpty()) {</span><br><span class="line"> mappedFile.setFirstCreateInQueue(<span class="literal">true</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">this</span>.mappedFiles.add(mappedFile);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> mappedFile;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> mappedFileLast;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>首先看下直接创建的，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">MappedFile</span><span class="params">(<span class="keyword">final</span> String fileName, <span class="keyword">final</span> <span class="type">int</span> fileSize)</span> <span class="keyword">throws</span> IOException {</span><br><span class="line"> init(fileName, fileSize);</span><br><span class="line"> }</span><br><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">init</span><span class="params">(<span class="keyword">final</span> String fileName, <span class="keyword">final</span> <span class="type">int</span> fileSize)</span> <span class="keyword">throws</span> IOException {</span><br><span class="line"> <span class="built_in">this</span>.fileName = fileName;</span><br><span class="line"> <span class="built_in">this</span>.fileSize = fileSize;</span><br><span class="line"> <span class="built_in">this</span>.file = <span class="keyword">new</span> <span class="title class_">File</span>(fileName);</span><br><span class="line"> <span class="built_in">this</span>.fileFromOffset = Long.parseLong(<span class="built_in">this</span>.file.getName());</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">ok</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"></span><br><span class="line"> ensureDirOK(<span class="built_in">this</span>.file.getParent());</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 通过 RandomAccessFile 创建 fileChannel</span></span><br><span class="line"> <span class="built_in">this</span>.fileChannel = <span class="keyword">new</span> <span class="title class_">RandomAccessFile</span>(<span class="built_in">this</span>.file, <span class="string">"rw"</span>).getChannel();</span><br><span class="line"> <span class="comment">// 做 mmap 映射</span></span><br><span class="line"> <span class="built_in">this</span>.mappedByteBuffer = <span class="built_in">this</span>.fileChannel.map(MapMode.READ_WRITE, <span class="number">0</span>, fileSize);</span><br><span class="line"> TOTAL_MAPPED_VIRTUAL_MEMORY.addAndGet(fileSize);</span><br><span class="line"> TOTAL_MAPPED_FILES.incrementAndGet();</span><br><span class="line"> ok = <span class="literal">true</span>;</span><br><span class="line"> } <span class="keyword">catch</span> (FileNotFoundException e) {</span><br><span class="line"> log.error(<span class="string">"create file channel "</span> + <span class="built_in">this</span>.fileName + <span class="string">" Failed. "</span>, e);</span><br><span class="line"> <span class="keyword">throw</span> e;</span><br><span class="line"> } <span class="keyword">catch</span> (IOException e) {</span><br><span class="line"> log.error(<span class="string">"map file "</span> + <span class="built_in">this</span>.fileName + <span class="string">" Failed. "</span>, e);</span><br><span class="line"> <span class="keyword">throw</span> e;</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="keyword">if</span> (!ok && <span class="built_in">this</span>.fileChannel != <span class="literal">null</span>) {</span><br><span class="line"> <span class="built_in">this</span>.fileChannel.close();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>如果是提交给<code>AllocateMappedFileService</code>的话就用到了一些异步操作</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> MappedFile <span class="title function_">putRequestAndReturnMappedFile</span><span class="params">(String nextFilePath, String nextNextFilePath, <span class="type">int</span> fileSize)</span> {</span><br><span class="line"> <span class="type">int</span> <span class="variable">canSubmitRequests</span> <span class="operator">=</span> <span class="number">2</span>;</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.messageStore.getMessageStoreConfig().isTransientStorePoolEnable()) {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.messageStore.getMessageStoreConfig().isFastFailIfNoBufferInStorePool()</span><br><span class="line"> && BrokerRole.SLAVE != <span class="built_in">this</span>.messageStore.getMessageStoreConfig().getBrokerRole()) { <span class="comment">//if broker is slave, don't fast fail even no buffer in pool</span></span><br><span class="line"> canSubmitRequests = <span class="built_in">this</span>.messageStore.getTransientStorePool().remainBufferNumbs() - <span class="built_in">this</span>.requestQueue.size();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 将请求放在 requestTable 中</span></span><br><span class="line"> <span class="type">AllocateRequest</span> <span class="variable">nextReq</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">AllocateRequest</span>(nextFilePath, fileSize);</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">nextPutOK</span> <span class="operator">=</span> <span class="built_in">this</span>.requestTable.putIfAbsent(nextFilePath, nextReq) == <span class="literal">null</span>;</span><br><span class="line"> <span class="comment">// requestTable 使用了 concurrentHashMap，用文件名作为 key，防止并发</span></span><br><span class="line"> <span class="keyword">if</span> (nextPutOK) {</span><br><span class="line"> <span class="comment">// 这里判断了是否可以提交到 TransientStorePool，涉及读写分离，后面再细聊</span></span><br><span class="line"> <span class="keyword">if</span> (canSubmitRequests <= <span class="number">0</span>) {</span><br><span class="line"> log.warn(<span class="string">"[NOTIFYME]TransientStorePool is not enough, so create mapped file error, "</span> +</span><br><span class="line"> <span class="string">"RequestQueueSize : {}, StorePoolSize: {}"</span>, <span class="built_in">this</span>.requestQueue.size(), <span class="built_in">this</span>.messageStore.getTransientStorePool().remainBufferNumbs());</span><br><span class="line"> <span class="built_in">this</span>.requestTable.remove(nextFilePath);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 塞到阻塞队列中</span></span><br><span class="line"> <span class="type">boolean</span> <span class="variable">offerOK</span> <span class="operator">=</span> <span class="built_in">this</span>.requestQueue.offer(nextReq);</span><br><span class="line"> <span class="keyword">if</span> (!offerOK) {</span><br><span class="line"> log.warn(<span class="string">"never expected here, add a request to preallocate queue failed"</span>);</span><br><span class="line"> }</span><br><span class="line"> canSubmitRequests--;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 这里的两个提交我猜测是为了多生成一个 CommitLog，</span></span><br><span class="line"> <span class="type">AllocateRequest</span> <span class="variable">nextNextReq</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">AllocateRequest</span>(nextNextFilePath, fileSize);</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">nextNextPutOK</span> <span class="operator">=</span> <span class="built_in">this</span>.requestTable.putIfAbsent(nextNextFilePath, nextNextReq) == <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">if</span> (nextNextPutOK) {</span><br><span class="line"> <span class="keyword">if</span> (canSubmitRequests <= <span class="number">0</span>) {</span><br><span class="line"> log.warn(<span class="string">"[NOTIFYME]TransientStorePool is not enough, so skip preallocate mapped file, "</span> +</span><br><span class="line"> <span class="string">"RequestQueueSize : {}, StorePoolSize: {}"</span>, <span class="built_in">this</span>.requestQueue.size(), <span class="built_in">this</span>.messageStore.getTransientStorePool().remainBufferNumbs());</span><br><span class="line"> <span class="built_in">this</span>.requestTable.remove(nextNextFilePath);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">offerOK</span> <span class="operator">=</span> <span class="built_in">this</span>.requestQueue.offer(nextNextReq);</span><br><span class="line"> <span class="keyword">if</span> (!offerOK) {</span><br><span class="line"> log.warn(<span class="string">"never expected here, add a request to preallocate queue failed"</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (hasException) {</span><br><span class="line"> log.warn(<span class="built_in">this</span>.getServiceName() + <span class="string">" service has exception. so return null"</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">AllocateRequest</span> <span class="variable">result</span> <span class="operator">=</span> <span class="built_in">this</span>.requestTable.get(nextFilePath);</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 这里就异步等着</span></span><br><span class="line"> <span class="keyword">if</span> (result != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">waitOK</span> <span class="operator">=</span> result.getCountDownLatch().await(waitTimeOut, TimeUnit.MILLISECONDS);</span><br><span class="line"> <span class="keyword">if</span> (!waitOK) {</span><br><span class="line"> log.warn(<span class="string">"create mmap timeout "</span> + result.getFilePath() + <span class="string">" "</span> + result.getFileSize());</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="built_in">this</span>.requestTable.remove(nextFilePath);</span><br><span class="line"> <span class="keyword">return</span> result.getMappedFile();</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> log.error(<span class="string">"find preallocate mmap failed, this never happen"</span>);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> log.warn(<span class="built_in">this</span>.getServiceName() + <span class="string">" service has exception. "</span>, e);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>而真正去执行文件操作的就是 <code>AllocateMappedFileService</code>的 run 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> log.info(<span class="built_in">this</span>.getServiceName() + <span class="string">" service started"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">while</span> (!<span class="built_in">this</span>.isStopped() && <span class="built_in">this</span>.mmapOperation()) {</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"> log.info(<span class="built_in">this</span>.getServiceName() + <span class="string">" service end"</span>);</span><br><span class="line"> }</span><br><span class="line"><span class="keyword">private</span> <span class="type">boolean</span> <span class="title function_">mmapOperation</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">isSuccess</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="type">AllocateRequest</span> <span class="variable">req</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 从阻塞队列里获取请求</span></span><br><span class="line"> req = <span class="built_in">this</span>.requestQueue.take();</span><br><span class="line"> <span class="type">AllocateRequest</span> <span class="variable">expectedRequest</span> <span class="operator">=</span> <span class="built_in">this</span>.requestTable.get(req.getFilePath());</span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == expectedRequest) {</span><br><span class="line"> log.warn(<span class="string">"this mmap request expired, maybe cause timeout "</span> + req.getFilePath() + <span class="string">" "</span></span><br><span class="line"> + req.getFileSize());</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (expectedRequest != req) {</span><br><span class="line"> log.warn(<span class="string">"never expected here, maybe cause timeout "</span> + req.getFilePath() + <span class="string">" "</span></span><br><span class="line"> + req.getFileSize() + <span class="string">", req:"</span> + req + <span class="string">", expectedRequest:"</span> + expectedRequest);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (req.getMappedFile() == <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">long</span> <span class="variable">beginTime</span> <span class="operator">=</span> System.currentTimeMillis();</span><br><span class="line"></span><br><span class="line"> MappedFile mappedFile;</span><br><span class="line"> <span class="keyword">if</span> (messageStore.getMessageStoreConfig().isTransientStorePoolEnable()) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 通过 transientStorePool 创建</span></span><br><span class="line"> mappedFile = ServiceLoader.load(MappedFile.class).iterator().next();</span><br><span class="line"> mappedFile.init(req.getFilePath(), req.getFileSize(), messageStore.getTransientStorePool());</span><br><span class="line"> } <span class="keyword">catch</span> (RuntimeException e) {</span><br><span class="line"> log.warn(<span class="string">"Use default implementation."</span>);</span><br><span class="line"> <span class="comment">// 默认创建</span></span><br><span class="line"> mappedFile = <span class="keyword">new</span> <span class="title class_">MappedFile</span>(req.getFilePath(), req.getFileSize(), messageStore.getTransientStorePool());</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// 默认创建</span></span><br><span class="line"> mappedFile = <span class="keyword">new</span> <span class="title class_">MappedFile</span>(req.getFilePath(), req.getFileSize());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">long</span> <span class="variable">eclipseTime</span> <span class="operator">=</span> UtilAll.computeEclipseTimeMilliseconds(beginTime);</span><br><span class="line"> <span class="keyword">if</span> (eclipseTime > <span class="number">10</span>) {</span><br><span class="line"> <span class="type">int</span> <span class="variable">queueSize</span> <span class="operator">=</span> <span class="built_in">this</span>.requestQueue.size();</span><br><span class="line"> log.warn(<span class="string">"create mappedFile spent time(ms) "</span> + eclipseTime + <span class="string">" queue size "</span> + queueSize</span><br><span class="line"> + <span class="string">" "</span> + req.getFilePath() + <span class="string">" "</span> + req.getFileSize());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// pre write mappedFile</span></span><br><span class="line"> <span class="keyword">if</span> (mappedFile.getFileSize() >= <span class="built_in">this</span>.messageStore.getMessageStoreConfig()</span><br><span class="line"> .getMapedFileSizeCommitLog()</span><br><span class="line"> &&</span><br><span class="line"> <span class="built_in">this</span>.messageStore.getMessageStoreConfig().isWarmMapedFileEnable()) {</span><br><span class="line"> mappedFile.warmMappedFile(<span class="built_in">this</span>.messageStore.getMessageStoreConfig().getFlushDiskType(),</span><br><span class="line"> <span class="built_in">this</span>.messageStore.getMessageStoreConfig().getFlushLeastPagesWhenWarmMapedFile());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> req.setMappedFile(mappedFile);</span><br><span class="line"> <span class="built_in">this</span>.hasException = <span class="literal">false</span>;</span><br><span class="line"> isSuccess = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> log.warn(<span class="built_in">this</span>.getServiceName() + <span class="string">" interrupted, possibly by shutdown."</span>);</span><br><span class="line"> <span class="built_in">this</span>.hasException = <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> } <span class="keyword">catch</span> (IOException e) {</span><br><span class="line"> log.warn(<span class="built_in">this</span>.getServiceName() + <span class="string">" service has exception. "</span>, e);</span><br><span class="line"> <span class="built_in">this</span>.hasException = <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> != req) {</span><br><span class="line"> requestQueue.offer(req);</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> Thread.sleep(<span class="number">1</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException ignored) {</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="keyword">if</span> (req != <span class="literal">null</span> && isSuccess)</span><br><span class="line"> <span class="comment">// 通知前面等待的</span></span><br><span class="line"> req.getCountDownLatch().countDown();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
|
|
|
|
]]></content>
|
|
<categories>
|
|
<category>MQ</category>
|
|
<category>RocketMQ</category>
|
|
<category>消息队列</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>MQ</tag>
|
|
<tag>消息队列</tag>
|
|
<tag>RocketMQ</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊一下 RocketMQ 的消息存储四</title>
|
|
<url>/2021/10/17/%E8%81%8A%E4%B8%80%E4%B8%8B-RocketMQ-%E7%9A%84%E6%B6%88%E6%81%AF%E5%AD%98%E5%82%A8%E5%9B%9B/</url>
|
|
<content><![CDATA[<p>IndexFile 结构 hash 结构能够通过 key 寻找到对应在 CommitLog 中的位置</p>
|
|
<p>IndexFile 的构建则是分发给这个进行处理</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">CommitLogDispatcherBuildIndex</span> <span class="keyword">implements</span> <span class="title class_">CommitLogDispatcher</span> {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">dispatch</span><span class="params">(DispatchRequest request)</span> {</span><br><span class="line"> <span class="keyword">if</span> (DefaultMessageStore.<span class="built_in">this</span>.messageStoreConfig.isMessageIndexEnable()) {</span><br><span class="line"> DefaultMessageStore.<span class="built_in">this</span>.indexService.buildIndex(request);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">buildIndex</span><span class="params">(DispatchRequest req)</span> {</span><br><span class="line"> <span class="type">IndexFile</span> <span class="variable">indexFile</span> <span class="operator">=</span> retryGetAndCreateIndexFile();</span><br><span class="line"> <span class="keyword">if</span> (indexFile != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">long</span> <span class="variable">endPhyOffset</span> <span class="operator">=</span> indexFile.getEndPhyOffset();</span><br><span class="line"> <span class="type">DispatchRequest</span> <span class="variable">msg</span> <span class="operator">=</span> req;</span><br><span class="line"> <span class="type">String</span> <span class="variable">topic</span> <span class="operator">=</span> msg.getTopic();</span><br><span class="line"> <span class="type">String</span> <span class="variable">keys</span> <span class="operator">=</span> msg.getKeys();</span><br><span class="line"> <span class="keyword">if</span> (msg.getCommitLogOffset() < endPhyOffset) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">tranType</span> <span class="operator">=</span> MessageSysFlag.getTransactionValue(msg.getSysFlag());</span><br><span class="line"> <span class="keyword">switch</span> (tranType) {</span><br><span class="line"> <span class="keyword">case</span> MessageSysFlag.TRANSACTION_NOT_TYPE:</span><br><span class="line"> <span class="keyword">case</span> MessageSysFlag.TRANSACTION_PREPARED_TYPE:</span><br><span class="line"> <span class="keyword">case</span> MessageSysFlag.TRANSACTION_COMMIT_TYPE:</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">case</span> MessageSysFlag.TRANSACTION_ROLLBACK_TYPE:</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (req.getUniqKey() != <span class="literal">null</span>) {</span><br><span class="line"> indexFile = putKey(indexFile, msg, buildKey(topic, req.getUniqKey()));</span><br><span class="line"> <span class="keyword">if</span> (indexFile == <span class="literal">null</span>) {</span><br><span class="line"> log.error(<span class="string">"putKey error commitlog {} uniqkey {}"</span>, req.getCommitLogOffset(), req.getUniqKey());</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (keys != <span class="literal">null</span> && keys.length() > <span class="number">0</span>) {</span><br><span class="line"> String[] keyset = keys.split(MessageConst.KEY_SEPARATOR);</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < keyset.length; i++) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">key</span> <span class="operator">=</span> keyset[i];</span><br><span class="line"> <span class="keyword">if</span> (key.length() > <span class="number">0</span>) {</span><br><span class="line"> indexFile = putKey(indexFile, msg, buildKey(topic, key));</span><br><span class="line"> <span class="keyword">if</span> (indexFile == <span class="literal">null</span>) {</span><br><span class="line"> log.error(<span class="string">"putKey error commitlog {} uniqkey {}"</span>, req.getCommitLogOffset(), req.getUniqKey());</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> log.error(<span class="string">"build index error, stop building index"</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>配置的数量</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="type">boolean</span> <span class="variable">messageIndexEnable</span> <span class="operator">=</span> <span class="literal">true</span>;</span><br><span class="line"><span class="keyword">private</span> <span class="type">int</span> <span class="variable">maxHashSlotNum</span> <span class="operator">=</span> <span class="number">5000000</span>;</span><br><span class="line"><span class="keyword">private</span> <span class="type">int</span> <span class="variable">maxIndexNum</span> <span class="operator">=</span> <span class="number">5000000</span> * <span class="number">4</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>最核心的其实是 IndexFile 的结构和如何写入</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">putKey</span><span class="params">(<span class="keyword">final</span> String key, <span class="keyword">final</span> <span class="type">long</span> phyOffset, <span class="keyword">final</span> <span class="type">long</span> storeTimestamp)</span> {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.indexHeader.getIndexCount() < <span class="built_in">this</span>.indexNum) {</span><br><span class="line"> <span class="comment">// 获取 key 的 hash</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">keyHash</span> <span class="operator">=</span> indexKeyHashMethod(key);</span><br><span class="line"> <span class="comment">// 计算属于哪个 slot</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">slotPos</span> <span class="operator">=</span> keyHash % <span class="built_in">this</span>.hashSlotNum;</span><br><span class="line"> <span class="comment">// 计算 slot 位置 因为结构是有个 indexHead，主要是分为三段 header，slot 和 index</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">absSlotPos</span> <span class="operator">=</span> IndexHeader.INDEX_HEADER_SIZE + slotPos * hashSlotSize;</span><br><span class="line"></span><br><span class="line"> <span class="type">FileLock</span> <span class="variable">fileLock</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"></span><br><span class="line"> <span class="comment">// fileLock = this.fileChannel.lock(absSlotPos, hashSlotSize,</span></span><br><span class="line"> <span class="comment">// false);</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">slotValue</span> <span class="operator">=</span> <span class="built_in">this</span>.mappedByteBuffer.getInt(absSlotPos);</span><br><span class="line"> <span class="keyword">if</span> (slotValue <= invalidIndex || slotValue > <span class="built_in">this</span>.indexHeader.getIndexCount()) {</span><br><span class="line"> slotValue = invalidIndex;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">long</span> <span class="variable">timeDiff</span> <span class="operator">=</span> storeTimestamp - <span class="built_in">this</span>.indexHeader.getBeginTimestamp();</span><br><span class="line"></span><br><span class="line"> timeDiff = timeDiff / <span class="number">1000</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.indexHeader.getBeginTimestamp() <= <span class="number">0</span>) {</span><br><span class="line"> timeDiff = <span class="number">0</span>;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (timeDiff > Integer.MAX_VALUE) {</span><br><span class="line"> timeDiff = Integer.MAX_VALUE;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (timeDiff < <span class="number">0</span>) {</span><br><span class="line"> timeDiff = <span class="number">0</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 计算索引存放位置，头部 + slot 数量 * slot 大小 + 已有的 index 数量 + index 大小</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">absIndexPos</span> <span class="operator">=</span></span><br><span class="line"> IndexHeader.INDEX_HEADER_SIZE + <span class="built_in">this</span>.hashSlotNum * hashSlotSize</span><br><span class="line"> + <span class="built_in">this</span>.indexHeader.getIndexCount() * indexSize;</span><br><span class="line"> </span><br><span class="line"> <span class="built_in">this</span>.mappedByteBuffer.putInt(absIndexPos, keyHash);</span><br><span class="line"> <span class="built_in">this</span>.mappedByteBuffer.putLong(absIndexPos + <span class="number">4</span>, phyOffset);</span><br><span class="line"> <span class="built_in">this</span>.mappedByteBuffer.putInt(absIndexPos + <span class="number">4</span> + <span class="number">8</span>, (<span class="type">int</span>) timeDiff);</span><br><span class="line"> <span class="built_in">this</span>.mappedByteBuffer.putInt(absIndexPos + <span class="number">4</span> + <span class="number">8</span> + <span class="number">4</span>, slotValue);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 存放的是数量位移，不是绝对位置</span></span><br><span class="line"> <span class="built_in">this</span>.mappedByteBuffer.putInt(absSlotPos, <span class="built_in">this</span>.indexHeader.getIndexCount());</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.indexHeader.getIndexCount() <= <span class="number">1</span>) {</span><br><span class="line"> <span class="built_in">this</span>.indexHeader.setBeginPhyOffset(phyOffset);</span><br><span class="line"> <span class="built_in">this</span>.indexHeader.setBeginTimestamp(storeTimestamp);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.indexHeader.incHashSlotCount();</span><br><span class="line"> <span class="built_in">this</span>.indexHeader.incIndexCount();</span><br><span class="line"> <span class="built_in">this</span>.indexHeader.setEndPhyOffset(phyOffset);</span><br><span class="line"> <span class="built_in">this</span>.indexHeader.setEndTimestamp(storeTimestamp);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> log.error(<span class="string">"putKey exception, Key: "</span> + key + <span class="string">" KeyHashCode: "</span> + key.hashCode(), e);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="keyword">if</span> (fileLock != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> fileLock.release();</span><br><span class="line"> } <span class="keyword">catch</span> (IOException e) {</span><br><span class="line"> log.error(<span class="string">"Failed to release the lock"</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> log.warn(<span class="string">"Over index file capacity: index count = "</span> + <span class="built_in">this</span>.indexHeader.getIndexCount()</span><br><span class="line"> + <span class="string">"; index max num = "</span> + <span class="built_in">this</span>.indexNum);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>具体可以看一下这个简略的示意图<br><img data-src="https://img.nicksxs.me/uPic/kqfruz.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>MQ</category>
|
|
<category>RocketMQ</category>
|
|
<category>消息队列</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>MQ</tag>
|
|
<tag>消息队列</tag>
|
|
<tag>RocketMQ</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊一下 SpringBoot 中使用的 cglib 作为动态代理中的一个注意点</title>
|
|
<url>/2021/09/19/%E8%81%8A%E4%B8%80%E4%B8%8B-SpringBoot-%E4%B8%AD%E4%BD%BF%E7%94%A8%E7%9A%84-cglib-%E4%BD%9C%E4%B8%BA%E5%8A%A8%E6%80%81%E4%BB%A3%E7%90%86%E4%B8%AD%E7%9A%84%E4%B8%80%E4%B8%AA%E6%B3%A8%E6%84%8F%E7%82%B9/</url>
|
|
<content><![CDATA[<p>这个话题是由一次组内同学分享引出来的，首先在 springboot 2.x 开始默认使用了 cglib 作为 aop 的实现，这里也稍微讲一下，在一个 1.x 的老项目里，可以看到AopAutoConfiguration 是这样的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Configuration</span></span><br><span class="line"><span class="meta">@ConditionalOnClass({ EnableAspectJAutoProxy.class, Aspect.class, Advice.class })</span></span><br><span class="line"><span class="meta">@ConditionalOnProperty(prefix = "spring.aop", name = "auto", havingValue = "true", matchIfMissing = true)</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">AopAutoConfiguration</span> {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Configuration</span></span><br><span class="line"> <span class="meta">@EnableAspectJAutoProxy(proxyTargetClass = false)</span></span><br><span class="line"> <span class="meta">@ConditionalOnProperty(prefix = "spring.aop", name = "proxy-target-class", havingValue = "false", matchIfMissing = true)</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">class</span> <span class="title class_">JdkDynamicAutoProxyConfiguration</span> {</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Configuration</span></span><br><span class="line"> <span class="meta">@EnableAspectJAutoProxy(proxyTargetClass = true)</span></span><br><span class="line"> <span class="meta">@ConditionalOnProperty(prefix = "spring.aop", name = "proxy-target-class", havingValue = "true", matchIfMissing = false)</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">class</span> <span class="title class_">CglibAutoProxyConfiguration</span> {</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>而在 2.x 中变成了这样</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Configuration(proxyBeanMethods = false)</span></span><br><span class="line"><span class="meta">@ConditionalOnProperty(prefix = "spring.aop", name = "auto", havingValue = "true", matchIfMissing = true)</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">AopAutoConfiguration</span> {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Configuration(proxyBeanMethods = false)</span></span><br><span class="line"> <span class="meta">@ConditionalOnClass(Advice.class)</span></span><br><span class="line"> <span class="keyword">static</span> <span class="keyword">class</span> <span class="title class_">AspectJAutoProxyingConfiguration</span> {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Configuration(proxyBeanMethods = false)</span></span><br><span class="line"> <span class="meta">@EnableAspectJAutoProxy(proxyTargetClass = false)</span></span><br><span class="line"> <span class="meta">@ConditionalOnProperty(prefix = "spring.aop", name = "proxy-target-class", havingValue = "false")</span></span><br><span class="line"> <span class="keyword">static</span> <span class="keyword">class</span> <span class="title class_">JdkDynamicAutoProxyConfiguration</span> {</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Configuration(proxyBeanMethods = false)</span></span><br><span class="line"> <span class="meta">@EnableAspectJAutoProxy(proxyTargetClass = true)</span></span><br><span class="line"> <span class="meta">@ConditionalOnProperty(prefix = "spring.aop", name = "proxy-target-class", havingValue = "true",</span></span><br><span class="line"><span class="meta"> matchIfMissing = true)</span></span><br><span class="line"> <span class="keyword">static</span> <span class="keyword">class</span> <span class="title class_">CglibAutoProxyConfiguration</span> {</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>为何会加载 AopAutoConfiguration 在前面的文章<a href="https://nicksxs.me/2021/07/11/%E8%81%8A%E8%81%8ASpringBoot-%E8%87%AA%E5%8A%A8%E8%A3%85%E9%85%8D/">聊聊 SpringBoot 自动装配</a>里已经介绍过，有兴趣的可以看下，可以发现 springboot 在 2.x 版本开始使用 cglib 作为默认的动态代理实现。</p>
|
|
<p>然后就是出现的问题了，代码是这样的，一个简单的基于 springboot 的带有数据库的插入，对插入代码加了事务注解，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Mapper</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">interface</span> <span class="title class_">StudentMapper</span> {</span><br><span class="line"> <span class="comment">// 就是插入一条数据</span></span><br><span class="line"> <span class="meta">@Insert("insert into student(name, age)" + "values ('nick', '18')")</span></span><br><span class="line"> <span class="keyword">public</span> Long <span class="title function_">insert</span><span class="params">()</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="meta">@Component</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">StudentManager</span> {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Resource</span></span><br><span class="line"> <span class="keyword">private</span> StudentMapper studentMapper;</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">public</span> Long <span class="title function_">createStudent</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> studentMapper.insert();</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="meta">@Component</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">StudentServiceImpl</span> <span class="keyword">implements</span> <span class="title class_">StudentService</span> {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Resource</span></span><br><span class="line"> <span class="keyword">private</span> StudentManager studentManager;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 自己引用</span></span><br><span class="line"> <span class="meta">@Resource</span></span><br><span class="line"> <span class="keyword">private</span> StudentServiceImpl studentService;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="meta">@Transactional</span></span><br><span class="line"> <span class="keyword">public</span> Long <span class="title function_">createStudent</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">Long</span> <span class="variable">id</span> <span class="operator">=</span> studentManager.createStudent();</span><br><span class="line"> <span class="type">Long</span> <span class="variable">id2</span> <span class="operator">=</span> studentService.createStudent2();</span><br><span class="line"> <span class="keyword">return</span> <span class="number">1L</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Transactional</span></span><br><span class="line"> <span class="keyword">private</span> Long <span class="title function_">createStudent2</span><span class="params">()</span> {</span><br><span class="line"><span class="comment">// Integer t = Integer.valueOf("aaa");</span></span><br><span class="line"> <span class="keyword">return</span> studentManager.createStudent();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>第一个公有方法 createStudent 首先调用了 manager 层的创建方法，然后再通过引入的 studentService 调用了createStudent2，我们先跑一下看看会出现啥情况，果不其然报错了，正是这个报错让我纠结了很久</p>
|
|
<p><img data-src="https://img.nicksxs.me/uPic/EdR7oB.png" alt="EdR7oB"></p>
|
|
<p>报了个空指针，而且是在 createStudent2 已经被调用到了，在它的内部，报的 studentManager 是 null，首先 cglib 作为动态代理它是通过继承的方式来实现的，相当于是会在调用目标对象的代理方法时调用 cglib 生成的子类，具体的代理切面逻辑在子类实现，然后在调用目标对象的目标方法，但是继承的方式对于 final 和私有方法其实是没法进行代理的，因为没法继承，所以我最开始的想法是应该通过 studentService 调用 createStudent2 的时候就报错了，也就是不会进入这个方法内部，后面才发现犯了个特别二的错误，继承的方式去调用父类的私有方法，对于 Java 来说是可以调用到的，父类的私有方法并不由子类的InstanceKlass维护，只能通过子类的InstanceKlass找到Java类对应的_super，这样间接地访问。也就是说子类其实是可以访问的，那为啥访问了会报空指针呢，这里报的是studentManager 是空的，可以往依赖注入方面去想，如果忽略依赖注入，我这个studentManager 的确是 null，那是不是就没有被依赖注入呢，但是为啥前面那个可以呢</p>
|
|
<p>这个问题着实查了很久，不废话来看代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">protected</span> Object <span class="title function_">invokeJoinpoint</span><span class="params">()</span> <span class="keyword">throws</span> Throwable {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.methodProxy != <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// 这里的 target 就是被代理的 bean</span></span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.methodProxy.invoke(<span class="built_in">this</span>.target, <span class="built_in">this</span>.arguments);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">super</span>.invokeJoinpoint();</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
|
|
<p>这个是<code>org.springframework.aop.framework.CglibAopProxy.CglibMethodInvocation</code>的代码，其实它在这里不是直接调用 super 也就是父类的方法，而是通过 methodProxy 调用 target 目标对象的方法，也就是原始的 studentService bean 的方法，这样子 spring 管理的已经做好依赖注入的 bean 就能正常起作用，否则就会出现上面的问题，因为 cglib 其实是通过继承来实现，通过将调用转移到子类上加入代理逻辑，我们在简单使用的时候会直接 invokeSuper() 调用父类的方法，但是在这里 spring 的场景里需要去支持 spring 的功能逻辑，所以上面的问题就可以开始来解释了，因为 createStudent 是公共方法，cglib 可以对其进行继承代理，但是在执行逻辑的时候其实是通过调用目标对象，也就是 spring 管理的被代理的目标对象的 bean 调用的 createStudent，而对于下面的 createStudent2 方法因为是私有方法，不会走代理逻辑，也就不会有调用回目标对象的逻辑，只是通过继承关系，在子类中没有这个方法，所以会通过子类的InstanceKlass找到这个类对应的_super，然后调用父类的这个私有方法，这里要搞清楚一个点，从这个代理类直接找到其父类然后调用这个私有方法，这个类是由 cglib 生成的，不是被 spring 管理起来经过依赖注入的 bean，所以是没有 studentManager 这个依赖的，也就出现了前面的问题</p>
|
|
<p>而在前面提到的cglib通过methodProxy调用到目标对象，目标对象是在什么时候设置的呢，其实是在bean的生命周期中，org.springframework.beans.factory.config.BeanPostProcessor#postProcessAfterInitialization这个接口的在bean的初始化过程中，会调用实现了这个接口的方法，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> Object <span class="title function_">postProcessAfterInitialization</span><span class="params">(<span class="meta">@Nullable</span> Object bean, String beanName)</span> {</span><br><span class="line"> <span class="keyword">if</span> (bean != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">Object</span> <span class="variable">cacheKey</span> <span class="operator">=</span> getCacheKey(bean.getClass(), beanName);</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.earlyProxyReferences.remove(cacheKey) != bean) {</span><br><span class="line"> <span class="keyword">return</span> wrapIfNecessary(bean, beanName, cacheKey);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> bean;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>具体的逻辑在 org.springframework.aop.framework.autoproxy.AbstractAutoProxyCreator#wrapIfNecessary这个方法里</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> Object <span class="title function_">getCacheKey</span><span class="params">(Class<?> beanClass, <span class="meta">@Nullable</span> String beanName)</span> {</span><br><span class="line"> <span class="keyword">if</span> (StringUtils.hasLength(beanName)) {</span><br><span class="line"> <span class="keyword">return</span> (FactoryBean.class.isAssignableFrom(beanClass) ?</span><br><span class="line"> BeanFactory.FACTORY_BEAN_PREFIX + beanName : beanName);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> beanClass;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Wrap the given bean if necessary, i.e. if it is eligible for being proxied.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> bean the raw bean instance</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> beanName the name of the bean</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> cacheKey the cache key for metadata access</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> a proxy wrapping the bean, or the raw bean instance as-is</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">protected</span> Object <span class="title function_">wrapIfNecessary</span><span class="params">(Object bean, String beanName, Object cacheKey)</span> {</span><br><span class="line"> <span class="keyword">if</span> (StringUtils.hasLength(beanName) && <span class="built_in">this</span>.targetSourcedBeans.contains(beanName)) {</span><br><span class="line"> <span class="keyword">return</span> bean;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (Boolean.FALSE.equals(<span class="built_in">this</span>.advisedBeans.get(cacheKey))) {</span><br><span class="line"> <span class="keyword">return</span> bean;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (isInfrastructureClass(bean.getClass()) || shouldSkip(bean.getClass(), beanName)) {</span><br><span class="line"> <span class="built_in">this</span>.advisedBeans.put(cacheKey, Boolean.FALSE);</span><br><span class="line"> <span class="keyword">return</span> bean;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Create proxy if we have advice.</span></span><br><span class="line"> Object[] specificInterceptors = getAdvicesAndAdvisorsForBean(bean.getClass(), beanName, <span class="literal">null</span>);</span><br><span class="line"> <span class="keyword">if</span> (specificInterceptors != DO_NOT_PROXY) {</span><br><span class="line"> <span class="built_in">this</span>.advisedBeans.put(cacheKey, Boolean.TRUE);</span><br><span class="line"> <span class="type">Object</span> <span class="variable">proxy</span> <span class="operator">=</span> createProxy(</span><br><span class="line"> bean.getClass(), beanName, specificInterceptors, <span class="keyword">new</span> <span class="title class_">SingletonTargetSource</span>(bean));</span><br><span class="line"> <span class="built_in">this</span>.proxyTypes.put(cacheKey, proxy.getClass());</span><br><span class="line"> <span class="keyword">return</span> proxy;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.advisedBeans.put(cacheKey, Boolean.FALSE);</span><br><span class="line"> <span class="keyword">return</span> bean;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后在 <code>org.springframework.aop.framework.autoproxy.AbstractAutoProxyCreator#createProxy</code> 中创建了代理类</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Spring</tag>
|
|
<tag>SpringBoot</tag>
|
|
<tag>cglib</tag>
|
|
<tag>事务</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊一下 SpringBoot 中动态切换数据源的方法</title>
|
|
<url>/2021/09/26/%E8%81%8A%E4%B8%80%E4%B8%8B-SpringBoot-%E4%B8%AD%E5%8A%A8%E6%80%81%E5%88%87%E6%8D%A2%E6%95%B0%E6%8D%AE%E6%BA%90%E7%9A%84%E6%96%B9%E6%B3%95/</url>
|
|
<content><![CDATA[<p>其实这个表示有点不太对，应该是 Druid 动态切换数据源的方法，只是应用在了 springboot 框架中，准备代码准备了半天，之前在一次数据库迁移中使用了，发现 Druid 还是很强大的，用来做动态数据源切换很方便。</p>
|
|
<p>首先这里的场景跟我原来用的有点点区别，在项目中使用的是通过配置中心控制数据源切换，统一切换，而这里的例子多加了个可以根据接口注解配置</p>
|
|
<p>第一部分是最核心的，如何基于 Spring JDBC 和 Druid 来实现数据源切换，是继承了<code>org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource</code> 这个类，他的<code>determineCurrentLookupKey</code>方法会被调用来获得用来决定选择那个数据源的对象，也就是 lookupKey，也可以通过这个类看到就是通过这个 lookupKey 来路由找到数据源。</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">DynamicDataSource</span> <span class="keyword">extends</span> <span class="title class_">AbstractRoutingDataSource</span> {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">protected</span> Object <span class="title function_">determineCurrentLookupKey</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (DatabaseContextHolder.getDatabaseType() != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> DatabaseContextHolder.getDatabaseType().getName();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> DatabaseType.MASTER1.getName();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>而如何使用这个 lookupKey 呢，就涉及到我们的 DataSource 配置了，原来就是我们可以直接通过spring 的 jdbc 配置数据源，像这样</p>
|
|
<p><img data-src="https://img.nicksxs.me/uPic/VQPrT3.png"></p>
|
|
<p>现在我们要使用 Druid 作为数据源了，然后配置 <code>DynamicDataSource </code>的参数，通过 key 来选择对应的 DataSource，也就是下面配的 master1 和 master2</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">bean</span> <span class="attr">id</span>=<span class="string">"master1"</span> <span class="attr">class</span>=<span class="string">"com.alibaba.druid.pool.DruidDataSource"</span> <span class="attr">init-method</span>=<span class="string">"init"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">destroy-method</span>=<span class="string">"close"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:driverClassName</span>=<span class="string">"com.mysql.cj.jdbc.Driver"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:url</span>=<span class="string">"${master1.demo.datasource.url}"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:username</span>=<span class="string">"${master1.demo.datasource.username}"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:password</span>=<span class="string">"${master1.demo.datasource.password}"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:initialSize</span>=<span class="string">"5"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:minIdle</span>=<span class="string">"1"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:maxActive</span>=<span class="string">"10"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:maxWait</span>=<span class="string">"60000"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:timeBetweenEvictionRunsMillis</span>=<span class="string">"60000"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:minEvictableIdleTimeMillis</span>=<span class="string">"300000"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:validationQuery</span>=<span class="string">"SELECT 'x'"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:testWhileIdle</span>=<span class="string">"true"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:testOnBorrow</span>=<span class="string">"false"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:testOnReturn</span>=<span class="string">"false"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:poolPreparedStatements</span>=<span class="string">"false"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:maxPoolPreparedStatementPerConnectionSize</span>=<span class="string">"20"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:connectionProperties</span>=<span class="string">"config.decrypt=true"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:filters</span>=<span class="string">"stat,config"</span>/></span></span><br><span class="line"></span><br><span class="line"> <span class="tag"><<span class="name">bean</span> <span class="attr">id</span>=<span class="string">"master2"</span> <span class="attr">class</span>=<span class="string">"com.alibaba.druid.pool.DruidDataSource"</span> <span class="attr">init-method</span>=<span class="string">"init"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">destroy-method</span>=<span class="string">"close"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:driverClassName</span>=<span class="string">"com.mysql.cj.jdbc.Driver"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:url</span>=<span class="string">"${master2.demo.datasource.url}"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:username</span>=<span class="string">"${master2.demo.datasource.username}"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:password</span>=<span class="string">"${master2.demo.datasource.password}"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:initialSize</span>=<span class="string">"5"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:minIdle</span>=<span class="string">"1"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:maxActive</span>=<span class="string">"10"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:maxWait</span>=<span class="string">"60000"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:timeBetweenEvictionRunsMillis</span>=<span class="string">"60000"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:minEvictableIdleTimeMillis</span>=<span class="string">"300000"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:validationQuery</span>=<span class="string">"SELECT 'x'"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:testWhileIdle</span>=<span class="string">"true"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:testOnBorrow</span>=<span class="string">"false"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:testOnReturn</span>=<span class="string">"false"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:poolPreparedStatements</span>=<span class="string">"false"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:maxPoolPreparedStatementPerConnectionSize</span>=<span class="string">"20"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:connectionProperties</span>=<span class="string">"config.decrypt=true"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">p:filters</span>=<span class="string">"stat,config"</span>/></span></span><br><span class="line"></span><br><span class="line"> <span class="tag"><<span class="name">bean</span> <span class="attr">id</span>=<span class="string">"dataSource"</span> <span class="attr">class</span>=<span class="string">"com.nicksxs.springdemo.config.DynamicDataSource"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"targetDataSources"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">map</span> <span class="attr">key-type</span>=<span class="string">"java.lang.String"</span>></span></span><br><span class="line"> <span class="comment"><!-- master --></span></span><br><span class="line"> <span class="tag"><<span class="name">entry</span> <span class="attr">key</span>=<span class="string">"master1"</span> <span class="attr">value-ref</span>=<span class="string">"master1"</span>/></span></span><br><span class="line"> <span class="comment"><!-- slave --></span></span><br><span class="line"> <span class="tag"><<span class="name">entry</span> <span class="attr">key</span>=<span class="string">"master2"</span> <span class="attr">value-ref</span>=<span class="string">"master2"</span>/></span></span><br><span class="line"> <span class="tag"></<span class="name">map</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">property</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"defaultTargetDataSource"</span> <span class="attr">ref</span>=<span class="string">"master1"</span>/></span></span><br><span class="line"> <span class="tag"></<span class="name">bean</span>></span></span><br></pre></td></tr></table></figure>
|
|
|
|
<p>现在就要回到头上，介绍下这个<code>DatabaseContextHolder</code>，这里使用了 ThreadLocal 存放这个 DatabaseType，为啥要用这个是因为前面说的我们想要让接口层面去配置不同的数据源，要把持相互隔离不受影响，就使用了 ThreadLocal，关于它也可以看我前面写的一篇文章<a href="https://nicksxs.me/2021/05/30/%E8%81%8A%E8%81%8A%E4%BC%A0%E8%AF%B4%E4%B8%AD%E7%9A%84-ThreadLocal/">聊聊传说中的 ThreadLocal</a>，而 DatabaseType 就是个简单的枚举</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">DatabaseContextHolder</span> {</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> ThreadLocal<DatabaseType> databaseTypeThreadLocal = <span class="keyword">new</span> <span class="title class_">ThreadLocal</span><>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> DatabaseType <span class="title function_">getDatabaseType</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> databaseTypeThreadLocal.get();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">putDatabaseType</span><span class="params">(DatabaseType databaseType)</span> {</span><br><span class="line"> databaseTypeThreadLocal.set(databaseType);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">clearDatabaseType</span><span class="params">()</span> {</span><br><span class="line"> databaseTypeThreadLocal.remove();</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"><span class="keyword">public</span> <span class="keyword">enum</span> <span class="title class_">DatabaseType</span> {</span><br><span class="line"> MASTER1(<span class="string">"master1"</span>, <span class="string">"1"</span>),</span><br><span class="line"> MASTER2(<span class="string">"master2"</span>, <span class="string">"2"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> String name;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> String value;</span><br><span class="line"></span><br><span class="line"> DatabaseType(String name, String value) {</span><br><span class="line"> <span class="built_in">this</span>.name = name;</span><br><span class="line"> <span class="built_in">this</span>.value = value;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> String <span class="title function_">getName</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> name;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> String <span class="title function_">getValue</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> value;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> DatabaseType <span class="title function_">getDatabaseType</span><span class="params">(String name)</span> {</span><br><span class="line"> <span class="keyword">if</span> (MASTER2.name.equals(name)) {</span><br><span class="line"> <span class="keyword">return</span> MASTER2;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> MASTER1;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这边可以看到就是通过动态地通过<code>putDatabaseType</code>设置<code>lookupKey</code>来进行数据源切换，要通过接口注解配置来进行设置的话，我们就需要一个注解</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Retention(RetentionPolicy.RUNTIME)</span></span><br><span class="line"><span class="meta">@Target(ElementType.METHOD)</span></span><br><span class="line"><span class="keyword">public</span> <span class="meta">@interface</span> DataSource {</span><br><span class="line"> String <span class="title function_">value</span><span class="params">()</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这个注解可以配置在我的接口方法上，比如这样</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">interface</span> <span class="title class_">StudentService</span> {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@DataSource("master1")</span></span><br><span class="line"> <span class="keyword">public</span> Student <span class="title function_">queryOne</span><span class="params">()</span>;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@DataSource("master2")</span></span><br><span class="line"> <span class="keyword">public</span> Student <span class="title function_">queryAnother</span><span class="params">()</span>;</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>通过切面来进行数据源的设置</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Aspect</span></span><br><span class="line"><span class="meta">@Component</span></span><br><span class="line"><span class="meta">@Order(-1)</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">DataSourceAspect</span> {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Pointcut("execution(* com.nicksxs.springdemo.service..*.*(..))")</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">pointCut</span><span class="params">()</span> {</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="meta">@Before("pointCut()")</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">before</span><span class="params">(JoinPoint point)</span></span><br><span class="line"> {</span><br><span class="line"> <span class="type">Object</span> <span class="variable">target</span> <span class="operator">=</span> point.getTarget();</span><br><span class="line"> System.out.println(target.toString());</span><br><span class="line"> <span class="type">String</span> <span class="variable">method</span> <span class="operator">=</span> point.getSignature().getName();</span><br><span class="line"> System.out.println(method);</span><br><span class="line"> Class<?>[] classz = target.getClass().getInterfaces();</span><br><span class="line"> Class<?>[] parameterTypes = ((MethodSignature) point.getSignature())</span><br><span class="line"> .getMethod().getParameterTypes();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">Method</span> <span class="variable">m</span> <span class="operator">=</span> classz[<span class="number">0</span>].getMethod(method, parameterTypes);</span><br><span class="line"> System.out.println(<span class="string">"method"</span>+ m.getName());</span><br><span class="line"> <span class="keyword">if</span> (m.isAnnotationPresent(DataSource.class)) {</span><br><span class="line"> <span class="type">DataSource</span> <span class="variable">data</span> <span class="operator">=</span> m.getAnnotation(DataSource.class);</span><br><span class="line"> System.out.println(<span class="string">"dataSource："</span>+data.value());</span><br><span class="line"> DatabaseContextHolder.putDatabaseType(DatabaseType.getDatabaseType(data.value()));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@After("pointCut()")</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">after</span><span class="params">()</span> {</span><br><span class="line"> DatabaseContextHolder.clearDatabaseType();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>通过接口判断是否带有注解跟是注解的值，DatabaseType 的配置不太好，不过先忽略了，然后在切点后进行清理</p>
|
|
<p>这是我 master1 的数据，</p>
|
|
<p><img data-src="https://img.nicksxs.me/uPic/sdQuJo.png"></p>
|
|
<p>master2 的数据</p>
|
|
<p><img data-src="https://img.nicksxs.me/uPic/fXK1Wk.png"></p>
|
|
<p>然后跑一下简单的 demo，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">(String...args)</span> {</span><br><span class="line"> LOGGER.info(<span class="string">"run here"</span>);</span><br><span class="line"> System.out.println(studentService.queryOne());</span><br><span class="line"> System.out.println(studentService.queryAnother());</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>看一下运行结果</p>
|
|
<p><img data-src="https://img.nicksxs.me/uPic/42VAeq.png"></p>
|
|
<p>其实这个方法应用场景不止可以用来迁移数据库，还能实现精细化的读写数据源分离之类的，算是做个简单记录和分享。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Spring</tag>
|
|
<tag>SpringBoot</tag>
|
|
<tag>Druid</tag>
|
|
<tag>数据源动态切换</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊一下 RocketMQ 的顺序消息</title>
|
|
<url>/2021/08/29/%E8%81%8A%E4%B8%80%E4%B8%8B-RocketMQ-%E7%9A%84%E9%A1%BA%E5%BA%8F%E6%B6%88%E6%81%AF/</url>
|
|
<content><![CDATA[<p>rocketmq 里有一种比较特殊的用法，就是顺序消息，比如订单的生命周期里，在创建，支付，签收等状态轮转中，会发出来对应的消息，这里面就比较需要去保证他们的顺序，当然在处理的业务代码也可以做对应的处理，结合消息重投，但是如果这里消息就能保证顺序性了，那么业务代码就能更好的关注业务代码的处理。</p>
|
|
<p>首先有一种情况是全局的有序，比如对于一个 topic 里就得发送线程保证只有一个，内部的 queue 也只有一个，消费线程也只有一个，这样就能比较容易的保证全局顺序性了，但是这里的问题就是完全限制了性能，不是很现实，在真实场景里很多都是比如对于同一个订单，需要去保证状态的轮转是按照预期的顺序来，而不必要全局的有序性。</p>
|
|
<p>对于这类的有序性，需要在发送和接收方都有对应的处理，在发送消息中，需要去指定 selector，即<code>MessageQueueSelector</code>，能够以固定的方式是分配到对应的 <code>MessageQueue</code></p>
|
|
<p>比如像 RocketMQ 中的示例</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">SendResult</span> <span class="variable">sendResult</span> <span class="operator">=</span> producer.send(msg, <span class="keyword">new</span> <span class="title class_">MessageQueueSelector</span>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> MessageQueue <span class="title function_">select</span><span class="params">(List<MessageQueue> mqs, Message msg, Object arg)</span> {</span><br><span class="line"> <span class="type">Long</span> <span class="variable">id</span> <span class="operator">=</span> (Long) arg; <span class="comment">//message queue is selected by #salesOrderID</span></span><br><span class="line"> <span class="type">long</span> <span class="variable">index</span> <span class="operator">=</span> id % mqs.size();</span><br><span class="line"> <span class="keyword">return</span> mqs.get((<span class="type">int</span>) index);</span><br><span class="line"> }</span><br><span class="line"> }, orderList.get(i).getOrderId());</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>而在消费侧有几个点比较重要，首先我们要保证一个 MessageQueue只被一个消费者消费，消费队列存在broker端，要保证 MessageQueue 只被一个消费者消费，那么消费者在进行消息拉取消费时就<strong>必须向mq服务器申请队列锁</strong>，消费者申请队列锁的代码存在于RebalanceService消息队列负载的实现代码中。</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">List<PullRequest> pullRequestList = <span class="keyword">new</span> <span class="title class_">ArrayList</span><PullRequest>();</span><br><span class="line"> <span class="keyword">for</span> (MessageQueue mq : mqSet) {</span><br><span class="line"> <span class="keyword">if</span> (!<span class="built_in">this</span>.processQueueTable.containsKey(mq)) {</span><br><span class="line"> <span class="comment">// 判断是否顺序，如果是顺序消费的，则需要加锁</span></span><br><span class="line"> <span class="keyword">if</span> (isOrder && !<span class="built_in">this</span>.lock(mq)) {</span><br><span class="line"> log.warn(<span class="string">"doRebalance, {}, add a new mq failed, {}, because lock failed"</span>, consumerGroup, mq);</span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.removeDirtyOffset(mq);</span><br><span class="line"> <span class="type">ProcessQueue</span> <span class="variable">pq</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ProcessQueue</span>();</span><br><span class="line"> <span class="type">long</span> <span class="variable">nextOffset</span> <span class="operator">=</span> <span class="built_in">this</span>.computePullFromWhere(mq);</span><br><span class="line"> <span class="keyword">if</span> (nextOffset >= <span class="number">0</span>) {</span><br><span class="line"> <span class="type">ProcessQueue</span> <span class="variable">pre</span> <span class="operator">=</span> <span class="built_in">this</span>.processQueueTable.putIfAbsent(mq, pq);</span><br><span class="line"> <span class="keyword">if</span> (pre != <span class="literal">null</span>) {</span><br><span class="line"> log.info(<span class="string">"doRebalance, {}, mq already exists, {}"</span>, consumerGroup, mq);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> log.info(<span class="string">"doRebalance, {}, add a new mq, {}"</span>, consumerGroup, mq);</span><br><span class="line"> <span class="type">PullRequest</span> <span class="variable">pullRequest</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">PullRequest</span>();</span><br><span class="line"> pullRequest.setConsumerGroup(consumerGroup);</span><br><span class="line"> pullRequest.setNextOffset(nextOffset);</span><br><span class="line"> pullRequest.setMessageQueue(mq);</span><br><span class="line"> pullRequest.setProcessQueue(pq);</span><br><span class="line"> pullRequestList.add(pullRequest);</span><br><span class="line"> changed = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> log.warn(<span class="string">"doRebalance, {}, add new mq failed, {}"</span>, consumerGroup, mq);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>在申请到锁之后会创建 pullRequest 进行消息拉取，消息拉取部分的代码实现在PullMessageService中，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> log.info(<span class="built_in">this</span>.getServiceName() + <span class="string">" service started"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">while</span> (!<span class="built_in">this</span>.isStopped()) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">PullRequest</span> <span class="variable">pullRequest</span> <span class="operator">=</span> <span class="built_in">this</span>.pullRequestQueue.take();</span><br><span class="line"> <span class="built_in">this</span>.pullMessage(pullRequest);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException ignored) {</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> log.error(<span class="string">"Pull Message Service Run Method exception"</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> log.info(<span class="built_in">this</span>.getServiceName() + <span class="string">" service end"</span>);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>消息拉取完后，需要提交到ConsumeMessageService中进行消费，顺序消费的实现为ConsumeMessageOrderlyService，提交消息进行消费的方法为ConsumeMessageOrderlyService#submitConsumeRequest，具体实现如下：</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">submitConsumeRequest</span><span class="params">(</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> List<MessageExt> msgs,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> ProcessQueue processQueue,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> MessageQueue messageQueue,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> <span class="type">boolean</span> dispathToConsume)</span> {</span><br><span class="line"> <span class="keyword">if</span> (dispathToConsume) {</span><br><span class="line"> <span class="type">ConsumeRequest</span> <span class="variable">consumeRequest</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ConsumeRequest</span>(processQueue, messageQueue);</span><br><span class="line"> <span class="built_in">this</span>.consumeExecutor.submit(consumeRequest);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>构建了一个ConsumeRequest对象，它是个实现了 runnable 接口的类，并提交给了线程池来并行消费，看下顺序消费的ConsumeRequest的run方法实现：</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.processQueue.isDropped()) {</span><br><span class="line"> log.warn(<span class="string">"run, the message queue not be able to consume, because it's dropped. {}"</span>, <span class="built_in">this</span>.messageQueue);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 获得 Consumer 消息队列锁，即单个线程独占</span></span><br><span class="line"> <span class="keyword">final</span> <span class="type">Object</span> <span class="variable">objLock</span> <span class="operator">=</span> messageQueueLock.fetchLockObject(<span class="built_in">this</span>.messageQueue);</span><br><span class="line"> <span class="keyword">synchronized</span> (objLock) {</span><br><span class="line"> <span class="comment">// (广播模式) 或者 (集群模式 && Broker消息队列锁有效)</span></span><br><span class="line"> <span class="keyword">if</span> (MessageModel.BROADCASTING.equals(ConsumeMessageOrderlyService.<span class="built_in">this</span>.defaultMQPushConsumerImpl.messageModel())</span><br><span class="line"> || (<span class="built_in">this</span>.processQueue.isLocked() && !<span class="built_in">this</span>.processQueue.isLockExpired())) {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">long</span> <span class="variable">beginTime</span> <span class="operator">=</span> System.currentTimeMillis();</span><br><span class="line"> <span class="comment">// 循环</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">boolean</span> <span class="variable">continueConsume</span> <span class="operator">=</span> <span class="literal">true</span>; continueConsume; ) {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.processQueue.isDropped()) {</span><br><span class="line"> log.warn(<span class="string">"the message queue not be able to consume, because it's dropped. {}"</span>, <span class="built_in">this</span>.messageQueue);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 消息队列分布式锁未锁定，提交延迟获得锁并消费请求</span></span><br><span class="line"> <span class="keyword">if</span> (MessageModel.CLUSTERING.equals(ConsumeMessageOrderlyService.<span class="built_in">this</span>.defaultMQPushConsumerImpl.messageModel())</span><br><span class="line"> && !<span class="built_in">this</span>.processQueue.isLocked()) {</span><br><span class="line"> log.warn(<span class="string">"the message queue not locked, so consume later, {}"</span>, <span class="built_in">this</span>.messageQueue);</span><br><span class="line"> ConsumeMessageOrderlyService.<span class="built_in">this</span>.tryLockLaterAndReconsume(<span class="built_in">this</span>.messageQueue, <span class="built_in">this</span>.processQueue, <span class="number">10</span>);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 消息队列分布式锁已经过期，提交延迟获得锁并消费请求</span></span><br><span class="line"> <span class="keyword">if</span> (MessageModel.CLUSTERING.equals(ConsumeMessageOrderlyService.<span class="built_in">this</span>.defaultMQPushConsumerImpl.messageModel())</span><br><span class="line"> && <span class="built_in">this</span>.processQueue.isLockExpired()) {</span><br><span class="line"> log.warn(<span class="string">"the message queue lock expired, so consume later, {}"</span>, <span class="built_in">this</span>.messageQueue);</span><br><span class="line"> ConsumeMessageOrderlyService.<span class="built_in">this</span>.tryLockLaterAndReconsume(<span class="built_in">this</span>.messageQueue, <span class="built_in">this</span>.processQueue, <span class="number">10</span>);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 当前周期消费时间超过连续时长，默认：60s，提交延迟消费请求。默认情况下，每消费1分钟休息10ms。</span></span><br><span class="line"> <span class="type">long</span> <span class="variable">interval</span> <span class="operator">=</span> System.currentTimeMillis() - beginTime;</span><br><span class="line"> <span class="keyword">if</span> (interval > MAX_TIME_CONSUME_CONTINUOUSLY) {</span><br><span class="line"> ConsumeMessageOrderlyService.<span class="built_in">this</span>.submitConsumeRequestLater(processQueue, messageQueue, <span class="number">10</span>);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 获取消费消息。此处和并发消息请求不同，并发消息请求已经带了消费哪些消息。</span></span><br><span class="line"> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">consumeBatchSize</span> <span class="operator">=</span></span><br><span class="line"> ConsumeMessageOrderlyService.<span class="built_in">this</span>.defaultMQPushConsumer.getConsumeMessageBatchMaxSize();</span><br><span class="line"></span><br><span class="line"> List<MessageExt> msgs = <span class="built_in">this</span>.processQueue.takeMessags(consumeBatchSize);</span><br><span class="line"> defaultMQPushConsumerImpl.resetRetryAndNamespace(msgs, defaultMQPushConsumer.getConsumerGroup());</span><br><span class="line"> <span class="keyword">if</span> (!msgs.isEmpty()) {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">ConsumeOrderlyContext</span> <span class="variable">context</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ConsumeOrderlyContext</span>(<span class="built_in">this</span>.messageQueue);</span><br><span class="line"></span><br><span class="line"> <span class="type">ConsumeOrderlyStatus</span> <span class="variable">status</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"></span><br><span class="line"> <span class="type">ConsumeMessageContext</span> <span class="variable">consumeMessageContext</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">if</span> (ConsumeMessageOrderlyService.<span class="built_in">this</span>.defaultMQPushConsumerImpl.hasHook()) {</span><br><span class="line"> consumeMessageContext = <span class="keyword">new</span> <span class="title class_">ConsumeMessageContext</span>();</span><br><span class="line"> consumeMessageContext</span><br><span class="line"> .setConsumerGroup(ConsumeMessageOrderlyService.<span class="built_in">this</span>.defaultMQPushConsumer.getConsumerGroup());</span><br><span class="line"> consumeMessageContext.setNamespace(defaultMQPushConsumer.getNamespace());</span><br><span class="line"> consumeMessageContext.setMq(messageQueue);</span><br><span class="line"> consumeMessageContext.setMsgList(msgs);</span><br><span class="line"> consumeMessageContext.setSuccess(<span class="literal">false</span>);</span><br><span class="line"> <span class="comment">// init the consume context type</span></span><br><span class="line"> consumeMessageContext.setProps(<span class="keyword">new</span> <span class="title class_">HashMap</span><String, String>());</span><br><span class="line"> ConsumeMessageOrderlyService.<span class="built_in">this</span>.defaultMQPushConsumerImpl.executeHookBefore(consumeMessageContext);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 执行消费</span></span><br><span class="line"> <span class="type">long</span> <span class="variable">beginTimestamp</span> <span class="operator">=</span> System.currentTimeMillis();</span><br><span class="line"> <span class="type">ConsumeReturnType</span> <span class="variable">returnType</span> <span class="operator">=</span> ConsumeReturnType.SUCCESS;</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">hasException</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="built_in">this</span>.processQueue.getLockConsume().lock(); <span class="comment">// 锁定处理队列</span></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.processQueue.isDropped()) {</span><br><span class="line"> log.warn(<span class="string">"consumeMessage, the message queue not be able to consume, because it's dropped. {}"</span>,</span><br><span class="line"> <span class="built_in">this</span>.messageQueue);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> status = messageListener.consumeMessage(Collections.unmodifiableList(msgs), context);</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> log.warn(<span class="string">"consumeMessage exception: {} Group: {} Msgs: {} MQ: {}"</span>,</span><br><span class="line"> RemotingHelper.exceptionSimpleDesc(e),</span><br><span class="line"> ConsumeMessageOrderlyService.<span class="built_in">this</span>.consumerGroup,</span><br><span class="line"> msgs,</span><br><span class="line"> messageQueue);</span><br><span class="line"> hasException = <span class="literal">true</span>;</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="built_in">this</span>.processQueue.getLockConsume().unlock(); <span class="comment">// 解锁</span></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == status</span><br><span class="line"> || ConsumeOrderlyStatus.ROLLBACK == status</span><br><span class="line"> || ConsumeOrderlyStatus.SUSPEND_CURRENT_QUEUE_A_MOMENT == status) {</span><br><span class="line"> log.warn(<span class="string">"consumeMessage Orderly return not OK, Group: {} Msgs: {} MQ: {}"</span>,</span><br><span class="line"> ConsumeMessageOrderlyService.<span class="built_in">this</span>.consumerGroup,</span><br><span class="line"> msgs,</span><br><span class="line"> messageQueue);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">long</span> <span class="variable">consumeRT</span> <span class="operator">=</span> System.currentTimeMillis() - beginTimestamp;</span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == status) {</span><br><span class="line"> <span class="keyword">if</span> (hasException) {</span><br><span class="line"> returnType = ConsumeReturnType.EXCEPTION;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> returnType = ConsumeReturnType.RETURNNULL;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (consumeRT >= defaultMQPushConsumer.getConsumeTimeout() * <span class="number">60</span> * <span class="number">1000</span>) {</span><br><span class="line"> returnType = ConsumeReturnType.TIME_OUT;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (ConsumeOrderlyStatus.SUSPEND_CURRENT_QUEUE_A_MOMENT == status) {</span><br><span class="line"> returnType = ConsumeReturnType.FAILED;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (ConsumeOrderlyStatus.SUCCESS == status) {</span><br><span class="line"> returnType = ConsumeReturnType.SUCCESS;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (ConsumeMessageOrderlyService.<span class="built_in">this</span>.defaultMQPushConsumerImpl.hasHook()) {</span><br><span class="line"> consumeMessageContext.getProps().put(MixAll.CONSUME_CONTEXT_TYPE, returnType.name());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="literal">null</span> == status) {</span><br><span class="line"> status = ConsumeOrderlyStatus.SUSPEND_CURRENT_QUEUE_A_MOMENT;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (ConsumeMessageOrderlyService.<span class="built_in">this</span>.defaultMQPushConsumerImpl.hasHook()) {</span><br><span class="line"> consumeMessageContext.setStatus(status.toString());</span><br><span class="line"> consumeMessageContext</span><br><span class="line"> .setSuccess(ConsumeOrderlyStatus.SUCCESS == status || ConsumeOrderlyStatus.COMMIT == status);</span><br><span class="line"> ConsumeMessageOrderlyService.<span class="built_in">this</span>.defaultMQPushConsumerImpl.executeHookAfter(consumeMessageContext);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> ConsumeMessageOrderlyService.<span class="built_in">this</span>.getConsumerStatsManager()</span><br><span class="line"> .incConsumeRT(ConsumeMessageOrderlyService.<span class="built_in">this</span>.consumerGroup, messageQueue.getTopic(), consumeRT);</span><br><span class="line"></span><br><span class="line"> continueConsume = ConsumeMessageOrderlyService.<span class="built_in">this</span>.processConsumeResult(msgs, status, context, <span class="built_in">this</span>);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> continueConsume = <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.processQueue.isDropped()) {</span><br><span class="line"> log.warn(<span class="string">"the message queue not be able to consume, because it's dropped. {}"</span>, <span class="built_in">this</span>.messageQueue);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> ConsumeMessageOrderlyService.<span class="built_in">this</span>.tryLockLaterAndReconsume(<span class="built_in">this</span>.messageQueue, <span class="built_in">this</span>.processQueue, <span class="number">100</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>获取到锁对象后，使用synchronized尝试申请线程级独占锁。</p>
|
|
<p><strong>如果加锁成功</strong>，同一时刻只有一个线程进行消息消费。</p>
|
|
<p><strong>如果加锁失败</strong>，会延迟100ms重新尝试向broker端申请锁定messageQueue，锁定成功后重新提交消费请求</p>
|
|
<p>创建消息拉取任务时，消息客户端向broker端申请锁定MessageQueue，使得一个MessageQueue同一个时刻只能被一个消费客户端消费。</p>
|
|
<p>消息消费时，多线程针对同一个消息队列的消费先尝试使用synchronized申请独占锁，加锁成功才能进行消费，使得一个MessageQueue同一个时刻只能被一个消费客户端中一个线程消费。<br>这里其实还有很重要的一点是对processQueue的加锁，这里其实是保证了在 rebalance的过程中如果 processQueue 被分配给了另一个 consumer，但是当前已经被我这个 consumer 再消费，但是没提交，就有可能出现被两个消费者消费，所以得进行加锁保证不受 rebalance 影响。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>MQ</category>
|
|
<category>RocketMQ</category>
|
|
<category>消息队列</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>MQ</tag>
|
|
<tag>消息队列</tag>
|
|
<tag>RocketMQ</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊一下 SpringBoot 设置非 web 应用的方法</title>
|
|
<url>/2022/07/31/%E8%81%8A%E4%B8%80%E4%B8%8B-SpringBoot-%E8%AE%BE%E7%BD%AE%E9%9D%9E-web-%E5%BA%94%E7%94%A8%E7%9A%84%E6%96%B9%E6%B3%95/</url>
|
|
<content><![CDATA[<h3 id="寻找原因"><a href="#寻找原因" class="headerlink" title="寻找原因"></a>寻找原因</h3><p>这次碰到一个比较奇怪的问题，应该统一发布脚本统一给应用启动参数传了个 <code>-Dserver.port=xxxx</code>，其实这个端口会作为 dubbo 的服务端口，并且应用也不提供 web 服务，但是在启动的时候会报<code>embedded servlet container failed to start. port xxxx was already in use</code>就觉得有点奇怪，仔细看了启动参数猜测可能是这个问题，有可能是依赖的二方三方包带了 spring-web 的包，然后基于 springboot 的 auto configuration 会把这个自己加载，就在本地复现了下这个问题，结果的确是这个问题。</p>
|
|
<h3 id="解决方案"><a href="#解决方案" class="headerlink" title="解决方案"></a>解决方案</h3><h4 id="老版本-设置-spring-不带-web-功能"><a href="#老版本-设置-spring-不带-web-功能" class="headerlink" title="老版本 设置 spring 不带 web 功能"></a>老版本 设置 spring 不带 web 功能</h4><p>比较老的 springboot 版本，可以使用 </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">SpringApplication</span> <span class="variable">app</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SpringApplication</span>(XXXXXApplication.class);</span><br><span class="line">app.setWebEnvironment(<span class="literal">false</span>);</span><br><span class="line">app.run(args);</span><br></pre></td></tr></table></figure>
|
|
<h4 id="新版本"><a href="#新版本" class="headerlink" title="新版本"></a>新版本</h4><p>新版本的 springboot (>= 2.0.0)可以在 properties 里配置</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">spring.main.web-application-type=none</span><br></pre></td></tr></table></figure>
|
|
<p>或者</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">SpringApplication</span> <span class="variable">app</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SpringApplication</span>(XXXXXApplication.class);</span><br><span class="line">app.setWebApplicationType(WebApplicationType.NONE);</span><br></pre></td></tr></table></figure>
|
|
<p>这个枚举里还有其他两种配置</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">enum</span> <span class="title class_">WebApplicationType</span> {</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * The application should not run as a web application and should not start an</span></span><br><span class="line"><span class="comment"> * embedded web server.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> NONE,</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * The application should run as a servlet-based web application and should start an</span></span><br><span class="line"><span class="comment"> * embedded servlet web server.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> SERVLET,</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * The application should run as a reactive web application and should start an</span></span><br><span class="line"><span class="comment"> * embedded reactive web server.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> REACTIVE</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>相当于是把none 的类型和包括 servlet 和 reactive 放进了枚举类进行控制。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Spring</tag>
|
|
<tag>SpringBoot</tag>
|
|
<tag>自动装配</tag>
|
|
<tag>AutoConfiguration</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊一下关于怎么陪伴学习</title>
|
|
<url>/2022/11/06/%E8%81%8A%E4%B8%80%E4%B8%8B%E5%85%B3%E4%BA%8E%E6%80%8E%E4%B9%88%E9%99%AA%E4%BC%B4%E5%AD%A6%E4%B9%A0/</url>
|
|
<content><![CDATA[<p>这是一次开车过程中结合网上的一些微博想到的，开车是之前LD买了车后，陪领导练车，其实在一开始练车的时候，我们已经是找了相对很空的封闭路段，路上基本很少有车，偶尔有一辆车，但是LD还是很害怕，车速还只有十几的时候，还很远的对面来车的时候就觉得很慌了，这个时候如果以常理肯定会说这样子完全不用怕，如果克服恐惧真的这么容易的话，问题就不会那么纠结了，人生是很难完全感同身受的，唯有降低预设的基准让事情从头理清楚，害怕了我们就先休息，有车了我们就停下，先适应完全没车的情况，变得更慢一点，如果这时候着急一点，反而会起到反效果，比如只是说不要怕，接着开，甚至有点厌烦了，那基本这个练车也不太成得了了，而正好是有耐心的一起慢慢练习，还有就是第二件是切身体会，就是当道路本来是两条道，但是封了一条的时候，这时候开车如果是像我这样的新手，如果开车时左右边看着的话，车肯定开不好，因为那样会一直左右调整，反而更容易控制不好左右的距离，蹭到旁边的隔离栏，正确的方式应该是专注于正前方的路，这样才能保证左右边距离尽可能均匀，而不是顾左失右或者顾右失左，所以很多陪伴学习需要注意的是方式和耐心，能够识别到关键点那是最好的，但是有时候更需要的是耐心，纯靠耐心不一定能解决问题，但是可能会找到问题关键点。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊在东京奥运会闭幕式这天-二</title>
|
|
<url>/2021/08/19/%E8%81%8A%E5%9C%A8%E4%B8%9C%E4%BA%AC%E5%A5%A5%E8%BF%90%E4%BC%9A%E9%97%AD%E5%B9%95%E5%BC%8F%E8%BF%99%E5%A4%A9-%E4%BA%8C/</url>
|
|
<content><![CDATA[<p>前面主要还是说了乒乓球的，因为整体还是乒乓球的比赛赛程比较长，比较激烈，扣人心弦，记得那会在公司没法看视频直播，就偶尔看看奥运会官网的比分，还几场马龙樊振东，陈梦被赢了一局就吓尿了，已经被混双那场留下了阴影，其实后面去看看16 年的比赛什么的，中国队虽然拿了这么多冠军，但是自改成 11 分制以来，其实都没办法那么完全彻底地碾压，而且像张继科，樊振东，陈梦都多少有些慢热，现在看来是马龙比较全面，不过看过了马龙，刘国梁，许昕等的一些过往经历，都是起起伏伏，即使是张怡宁这样的大魔王，也经历过逢王楠不赢的阶段，心态无法调整好。</p>
|
|
<p>其实最开始是举重项目，侯志慧是女子 49 公斤级的冠军，这场比赛是全场都看，其实看中国队的举重比赛跟跳水有点像，每一轮都需要到最后才能等到中国队，跳水其实每轮都有，举重会按照自己报的试举重量进行排名，重量大的会在后面举，抓举和挺举各三次试举机会，有时候会看着比较焦虑，一直等不来，怕一上来就没试举成功，而且中国队一般试举重量就是很大的，容易一次试举不成功就马上下一次，连着举其实压力会非常大，说实话真的是外行看热闹，每次都是多懂一点点，这次由于实在是比较无聊，所以看的会比较专心点，对于对应的规则知识点也会多了解一点，同时对于举重，没想到我们国家的这些运动员有这么强，最后八块金牌拿了七块，有一块拿到银牌也是有点因为教练的策略问题，这里其实也稍微知道一点，因为报上去的试举重量是谁小谁先举，并且我们国家都是实力非常强的，所以都会报大一些，并且如果这个项目有实力相近的选手，会比竞对多报一公斤，这样子如果前面竞争对手没举成功，我们把握就很大了，最坏的情况即使对手试举成功了，我们还有机会搏一把，比如谌利军这样的，只是说说感想，举重运动员真的是个比较单纯的群体，而且训练是非常痛苦枯燥的，非常容易受伤，像挺举就有点会压迫呼吸通道，看到好几个都是脸憋得通红，甚至直接因为压迫气道而没法完成后面的挺举，像之前 16 年的举重比赛，有个运动员没成功夺冠就非常愧疚地哭着说对不起祖国，没有获得冠军，这是怎么样的一种歉疚，怎么样的一种纯粹的感情呢，相对应地来说，我又要举男足，男篮的例子了，很多人在那嘲笑我这样对男足男篮愤愤不平的人，说可能我这样的人都没交个税（从缴纳个税的数量比例来算有可能），只是这里有两个打脸的事情，我足额缴纳个税，接近 20%的薪资都缴了个税，并且我买的所有东西都缴了增值税，如果让我这样缴纳了个税，缴纳了增值税的有个人的投票权，我一定会投票不让男足男篮使用我缴纳我的税金，用我们的缴纳的税，打出这么烂的表现，想乒乓球混双，拿个亚军都会被喷，那可是世界第二了，而且是就输了那么一场，足球篮球呢，我觉得是一方面成绩差，因为比赛真的有状态跟心态的影响，偶尔有一场失误非常正常，NBA 被黑八的有这么多强队，但是如果像男足男篮，成绩是越来越差，用范志毅的话来说就是脸都不要了，还有就是精气神，要在比赛中打出胜负欲，保持这种争胜心，才有机会再进步，前火箭队主教练鲁迪·汤姆贾诺维奇的话，“永远不要低估冠军的决心”，即使我现在打不过你，我会在下一次，下下次打败你，竞技体育永远要有这种精神，可以接受一时的失败，但是要保持永远争胜的心。</p>
|
|
<p>第一块金牌是杨倩拿下的，中国队拿奥运会首金也是有政治任务的，而恰恰杨倩这个金牌也有点碰巧是对手最后一枪失误了，当然竞技体育，特别是射击，真的是容不得一点点失误，像前面几届的美国神通埃蒙斯，失之毫厘差之千里，但是这个具体评价就比较少，唯一一点让我比较出戏的就是杨倩真的非常像王刚的徒弟漆二娃，哈哈，微博上也有挺多人觉得像，射击还是个比较可以接受年纪稍大的运动员，需要经验和稳定性，相对来说爆发力体力稍好一点，像庞伟这样的，混合团体10米气手枪金牌，36 岁可能其他项目已经是年龄很大了，不过前面说的举重的吕小军军神也是年纪蛮大了，但是非常强，而且在油管上简直就是个神，相对来说射击是关注比较少，杨倩的也只是看了后面拿到冠军这个结果，有些因为时间或者电视上没放，但是成绩还是不错的，没多少喷点。</p>
|
|
<p>第二篇先到这，纯主观，轻喷。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>运动</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>运动</tag>
|
|
<tag>东京奥运会</tag>
|
|
<tag>举重</tag>
|
|
<tag>射击</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊一下 Linux 中 dmesg 跟 journal 的差别</title>
|
|
<url>/2024/07/14/%E8%81%8A%E4%B8%80%E4%B8%8B-Linux-%E4%B8%AD-dmesg-%E8%B7%9F-journal-%E7%9A%84%E5%B7%AE%E5%88%AB/</url>
|
|
<content><![CDATA[<p>最近在迁移一个自己用的mysql实例，发现用 portainer 安装 mysql 一直失败，还以为是配置了自定义端口映射被系统防火墙限制，但后面不映射端口也是不行，一开始查看 journal日志也没什么发现，因为之前在腾讯云的小机器也是正常的部署，所以没觉得是可能会oom，一时间就有点没头绪，结果也是好奇就看了下dmesg，发现里面都是oom，给了1g的内存的ubuntu虚拟机，安装了一两个docker就不行了，不过这里比较重要的就是记录下dmesg 跟 journal日志的这点区别<br>dmesg 打印了内核的日志缓冲，所以这个跟journal差别的第一点就是一个是个会被刷掉的缓冲区，另一个是记在文件，但更重要的是dmesg是打印的内核日志，而journal是会通过systemd收集日志，journal可以收集内核日志，但是可以进行配置，并且可能会晚于dmesg<br><img data-src="https://img.nicksxs.me/blog/SRF8P3.png"><br>而对于这个oom，如果是系统的oom killer，那的确是有可能dmesg会能查看到，而journal看不到<br>简单记录下，PS：奶娃是有点累，一天都没啥时间了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>linux</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>linux</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊在东京奥运会闭幕式这天</title>
|
|
<url>/2021/08/08/%E8%81%8A%E5%9C%A8%E4%B8%9C%E4%BA%AC%E5%A5%A5%E8%BF%90%E4%BC%9A%E9%97%AD%E5%B9%95%E5%BC%8F%E8%BF%99%E5%A4%A9/</url>
|
|
<content><![CDATA[<p>这届奥运会有可能是我除了 08 年之外关注度最高的一届奥运会，原因可能是因为最近也没什么电影综艺啥的比较好看，前面看跑男倒还行，不是说多好，也就图一乐，最开始看向往的生活觉得也挺不错的，后面变成了统一来了就看黄磊做饭，然后夸黄磊做饭好吃，然后无聊的说这种生活多么多么美好，单调无聊，差不多弃了，这里面还包括大华不在了，大华其实个人还是有点呱噪的，但是挺能搞气氛，并且也有才华，彭彭跟子枫人是不讨厌，但是撑不起来，所以也导致了前面说的结果，都变成了黄磊彩虹屁现场，虽然偶尔怀疑他是否做得好吃，但是整体还是承认的，可对于一个这么多季了的综艺来说，这样也有点单调了。</p>
|
|
<p>还有奥运会像乒乓球，篮球，跳水这几个都是比较喜欢的项目，篮球🏀是从初中开始就也有在自己在玩的，虽然因为身高啊体质基本没什么天赋，但也算是热爱驱动，差不多到了大学因为比较懒才放下了，初中高中还是有很多时间花在上面，不像别人经常打球跑跑跳跳还能长高，我反而一直都没长个子，也因为这个其实蛮遗憾的，后面想想可能是初中的时候远走他乡去住宿读初中，伙食营养跟不上导致的，可能也是自己的一厢情愿吧，总觉得应该还能再长点个，这一点以后我自己的小孩我应该会特别注意这段时间他/她的营养摄入了；然后像乒乓球🏓的话其实小时候是比较讨厌的，因为家里人，父母都没有这类爱好习惯，我也完全不会，但是小学那会班里的“恶霸”就以公平之名要我们男生每个人都排队打几个，我这种不会的反而又要被嘲笑，这个小时候的阴影让我有了比较不好的印象，对它🏓的改观是在工作以后，前司跟一个同样不会的同事经常在饭点会打打，而且那会因为这个其实身体得到了锻炼，感觉是个不错的健身方式，然后又是中国的优势项目，小时候跟着我爸看孔令辉，那时候完全不懂，印象就觉得老瓦很牛，后面其实也没那么关注，上一届好像看了马龙的比赛；跳水也是中国的优势项目，而且也比较简单，不是说真的很简单，就是我们外行观众看着就看看水花大小图一乐。</p>
|
|
<p>这次的观赛过程其实主要还是在乒乓球上面，现在都有点怪我的乌鸦嘴，混双我一直就不太放心（关我什么事，我也不专业），然后一直觉得混双是不是不太稳，结果那天看的时候也是因为央视一套跟五套都没放，我家的有线电视又是没有五加体育，然后用电脑投屏就很卡，看得也很不爽，同时那天因为看的时候已经是 2:0还是再后面点了，一方面是不懂每队只有一次暂停，另一方面不知道已经用过暂停了，所以就特别怀疑马林是不是只会无脑鼓掌，感觉作为教练，并且是前冠军，应该也能在擦汗间隙，或者局间休息调整的时候多给些战略战术的指导，类似于后面男团小胖打奥恰洛夫，像解说都看出来了，其实奥恰那会的反手特别顺，打得特别凶，那就不能让他能特别顺手的上反手位，这当然是外行比较粗浅的看法，在混双过程中其实除了这个，还有让人很不爽的就是我们的许昕跟刘诗雯有种拿不出破釜沉舟的勇气的感觉，在气势上完全被对面两位日本乒乓球最讨厌的两位对手压制着，我都要输了，我就每一颗都要不让你好过，因为真的不是说没有实力，对面水谷隼也不是多么多么强的，可能上一届男团许昕输给他还留着阴影，但是以许昕 19 年男单世界第一的实力，目前也排在世界前三，输一场不应该成为这种阻力，有一些失误也很可惜，后面孙颖莎真的打得很解气，第二局一度以为又要被翻盘了，结果来了个大逆转，女团的时候也是，感觉在心态上孙颖莎还是很值得肯定的，少年老成这个词很适合，看其他的视频也觉得莎莎萌萌哒，陈梦总感觉还欠一点王者霸气，王曼昱还是可以的，反手很凶，我觉得其实这一届日本女乒就是打得非常凶，即使像平野这种看着很弱的妹子，打的球可一点都不弱，也是这种凶狠的打法，有点要压制中国的感觉，这方面我觉得是需要改善的，打这种要不就是实力上的完全碾压，要不就是我实力虽然比较没强多少，但是你狠我打得比你还狠，越保守越要输，我不太成熟的想法是这样的，还有就是面对逆境，这个就要说到男队的了，樊振东跟马龙在半决赛的时候，特别是男团的第二盘，樊振东打奥恰很好地表现了这个心态，当然樊振东我不是特别了解，据说他是比较善于打相持，比较善于焦灼的情况，不过整体看下来樊振东还是有一些欠缺，就是面对情况的快速转变应对，这一点也是马龙特别强的，虽然看起来马龙真的是年纪大了点，没有 16 年那会满头发胶，油光锃亮的大背头和满脸胶原蛋白的意气风发，大范围运动能力也弱了一点，但是经验和能力的全面性也让他最终能再次站上巅峰，还是非常佩服的，这里提一下张继科，虽然可能天赋上是张继科更强点，但是男乒一直都是有强者出现，能为国家队付出这么多并且一直坚持的可不是人人都可以，即使现在同台竞技马龙打不过张继科我还是更喜欢马龙。再来说说我们的对手，主要分三部分，德国男乒，里面有波尔（我刚听到的时候在想怎么又出来个叫波尔的，是不是像举重的石智勇一样，又来一个同名的，结果是同一个，已经四十岁了），这真是个让人敬佩的对手，实力强，经验丰富，虽然男单有点可惜，但是帮助男团获得银牌，真的是起到了定海神针的作用；奥恰洛夫，以前完全不认识，或者说看过也忘了，这次是真的有点意外，竟然有这么个马龙护法，其实他也坦言非常想赢一次马龙，并且在半决赛也非常接近赢得比赛，是个实力非常强的对手，就是男团半决赛输给张本智和有点可惜，有点被打蒙的感觉，佛朗西斯卡的话也是实力不错的选手，就是可能被奥恰跟波尔的光芒掩盖了，跟波尔在男团第一盘男双的比赛中打败日本那对男双也是非常给力的，说实话，最后打国乒的时候的确是国乒实力更胜一筹，但是即使德国赢了我也是充满尊敬，拼的就是硬实力，就像第二盘奥恰打樊振东，反手是真的很强，反过来看奥恰可能也不是很善于快速调整，樊振东打出来自己的节奏，主攻奥恰的中路，他好像没什么好办法解决。再来说我最讨厌的日本，嗯，小日本，张本智和、水谷隼、伊藤美诚，一一评价下（我是外行，绝对主观评价），张本智和，父母也是中国人，原来叫张智和，改日本籍后加了个本，被微博网友笑称日本尖叫鸡，男单输给了斯洛文尼亚选手，男团里是赢了两场，但是在我看来其实实力上可能比不上全力的奥恰，主要是特别能叫，会干扰对手，如果觉得这种也是种能力我也无话可说，要是有那种吼声能直接把对手震聋的，都不需要打比赛了，我简单记了下，赢一颗球，他要叫八声，用 LD 的话来说烦都烦死了，心态是在面对一些困境顺境的应对调整适应能力，而不是对这种噪音的适应能力，至少我是这么看的，所以我很期待樊振东能好好地虐虐他，因为其他像林昀儒真的是非常优秀的新选手，所谓的国乒克星估计也是小日本自己说说的，国乒其实有很多对手，马龙跟樊振东在男单半决赛碰到的这两个几乎都差点把他们掀翻了，所以还是练好自己的实力再来吹吧，免得打脸；水谷隼的话真的是长相就是特别地讨厌，还搞出那套不打比赛的姿态，男团里被波尔干掉就是很好的例子，波尔虽然真的很强，但毕竟 40 岁了，跟伊藤美诚一起说了吧，伊藤实力说实话是有的，混双中很大一部分的赢面来自于她，刘诗雯做了手术状态不好，许昕失误稍多，但是这种赢球了就感觉我赢了你一辈子一场没输的感觉，还有那种不知道怎么形容的笑，实力强的正常打比赛的我都佩服，像女团决赛里，平野跟石川佳纯的打法其实也很凶狠，但是都是正常的比赛，即使中国队两位实力不济输了也很正常，这种就真的需要像孙颖莎这样的小魔王无视各种魔法攻击，无视你各种花里胡哨的打法的人好好教训一下，混双输了以后了解了下她，感觉实力真的不错，是个大威胁，但是其实我们孙颖莎也是经历了九个月的继续成长，像张怡宁也评价了她，可能后面就没什么空间了，当然如果由张怡宁来打她就更适合了，净整这些有的没的，就打得你没脾气。</p>
|
|
<p>乒乓球的说的有点多，就分篇说了，第一篇先到这。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>运动</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>运动</tag>
|
|
<tag>东京奥运会</tag>
|
|
<tag>乒乓球</tag>
|
|
<tag>跳水</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 Dubbo 的 SPI</title>
|
|
<url>/2020/05/31/%E8%81%8A%E8%81%8A-Dubbo-%E7%9A%84-SPI/</url>
|
|
<content><![CDATA[<p>SPI全称是Service Provider Interface，咋眼看跟api是不是有点相似，api是application interface，这两个其实在某些方面有类似的地方，也有蛮大的区别，比如我们基于 dubbo 的微服务，一般我们可以提供服务，然后非泛化调用的话，我们可以把 api 包提供给应用调用方，他们根据接口签名传对应参数并配置好对应的服务发现如 zk 等就可以调用我们的服务了，然后 spi 会有点类似但是是反过来的关系，相当于是一种规范，比如我约定完成这个功能需要两个有两个接口，一个是连接的，一个是断开的，其实就可以用 jdbc 的驱动举例，比较老套了，然后各个厂家去做具体的实现吧，到时候根据我接口的全限定名的文件来加载实际的实现类，然后运行的时候调用对应实现类的方法就完了</p>
|
|
<p><img data-src="https://mystore-1255223546.cos.ap-chengdu.myqcloud.com/uPic/3sKdpg.png" alt="3sKdpg"></p>
|
|
<p>看上面的图，<code>java.sql.Driver</code>就是 spi，对应在classpath 的 META-INF/services 目录下的这个文件，里边的内容就是具体的实现类</p>
|
|
<p><img data-src="https://mystore-1255223546.cos.ap-chengdu.myqcloud.com/uPic/1590735097909.jpg" alt="1590735097909"></p>
|
|
<p>简单介绍了 Java的 SPI，再来说说 dubbo 的，dubbo 中为啥要用 SPI 呢，主要是为了框架的可扩展性和性能方面的考虑，比如协议层 dubbo 默认使用 dubbo 协议，同时也支持很多其他协议，也支持用户自己实现协议，那么跟 Java 的 SPI 会有什么区别呢，我们也来看个文件</p>
|
|
<p><img data-src="https://mystore-1255223546.cos.ap-chengdu.myqcloud.com/uPic/bqxWMp.png" alt="bqxWMp"></p>
|
|
<p>是不是看着很想，又有点不一样，在 Java 的 SPI 配置文件里每一行只有一个实现类的全限定名，在 Dubbo的 SPI配置文件中是 key=value 的形式，我们只需要对应的 key 就能加载对应的实现，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 返回指定名字的扩展。如果指定名字的扩展不存在，则抛异常 {<span class="doctag">@link</span> IllegalStateException}.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> name</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="meta">@SuppressWarnings("unchecked")</span></span><br><span class="line"> <span class="keyword">public</span> T <span class="title function_">getExtension</span><span class="params">(String name)</span> {</span><br><span class="line"> <span class="keyword">if</span> (name == <span class="literal">null</span> || name.length() == <span class="number">0</span>)</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalArgumentException</span>(<span class="string">"Extension name == null"</span>);</span><br><span class="line"> <span class="keyword">if</span> (<span class="string">"true"</span>.equals(name)) {</span><br><span class="line"> <span class="keyword">return</span> getDefaultExtension();</span><br><span class="line"> }</span><br><span class="line"> Holder<Object> holder = cachedInstances.get(name);</span><br><span class="line"> <span class="keyword">if</span> (holder == <span class="literal">null</span>) {</span><br><span class="line"> cachedInstances.putIfAbsent(name, <span class="keyword">new</span> <span class="title class_">Holder</span><Object>());</span><br><span class="line"> holder = cachedInstances.get(name);</span><br><span class="line"> }</span><br><span class="line"> <span class="type">Object</span> <span class="variable">instance</span> <span class="operator">=</span> holder.get();</span><br><span class="line"> <span class="keyword">if</span> (instance == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">synchronized</span> (holder) {</span><br><span class="line"> instance = holder.get();</span><br><span class="line"> <span class="keyword">if</span> (instance == <span class="literal">null</span>) {</span><br><span class="line"> instance = createExtension(name);</span><br><span class="line"> holder.set(instance);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> (T) instance;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里其实就可以看出来第二个不同点了，就是这个<code>cachedInstances</code>，第一个是不用像 Java 原生的 SPI 那样去遍历加载对应的服务类，只需要通过 key 去寻找，并且寻找的时候会先从缓存的对象里去取，还有就是注意下这里的 DCL(double check lock)</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@SuppressWarnings("unchecked")</span></span><br><span class="line"> <span class="keyword">private</span> T <span class="title function_">createExtension</span><span class="params">(String name)</span> {</span><br><span class="line"> Class<?> clazz = getExtensionClasses().get(name);</span><br><span class="line"> <span class="keyword">if</span> (clazz == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> findException(name);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">T</span> <span class="variable">instance</span> <span class="operator">=</span> (T) EXTENSION_INSTANCES.get(clazz);</span><br><span class="line"> <span class="keyword">if</span> (instance == <span class="literal">null</span>) {</span><br><span class="line"> EXTENSION_INSTANCES.putIfAbsent(clazz, (T) clazz.newInstance());</span><br><span class="line"> instance = (T) EXTENSION_INSTANCES.get(clazz);</span><br><span class="line"> }</span><br><span class="line"> injectExtension(instance);</span><br><span class="line"> Set<Class<?>> wrapperClasses = cachedWrapperClasses;</span><br><span class="line"> <span class="keyword">if</span> (wrapperClasses != <span class="literal">null</span> && wrapperClasses.size() > <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">for</span> (Class<?> wrapperClass : wrapperClasses) {</span><br><span class="line"> instance = injectExtension((T) wrapperClass.getConstructor(type).newInstance(instance));</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> instance;</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable t) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalStateException</span>(<span class="string">"Extension instance(name: "</span> + name + <span class="string">", class: "</span> +</span><br><span class="line"> type + <span class="string">") could not be instantiated: "</span> + t.getMessage(), t);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后就是创建扩展了，这里如果 wrapperClasses 就会遍历生成wrapper实例，并做 setter 依赖注入，但是这里cachedWrapperClasses的来源还是有点搞不清楚，得再看下 com.alibaba.dubbo.common.extension.ExtensionLoader#loadFile的具体逻辑<br>又看了遍新的代码，这个函数被抽出来了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * test if clazz is a wrapper class</span></span><br><span class="line"><span class="comment"> * <p></span></span><br><span class="line"><span class="comment"> * which has Constructor with given class type as its only argument</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">private</span> <span class="type">boolean</span> <span class="title function_">isWrapperClass</span><span class="params">(Class<?> clazz)</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> clazz.getConstructor(type);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> } <span class="keyword">catch</span> (NoSuchMethodException e) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>是否是 wrapperClass 其实就看构造函数的。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Dubbo</category>
|
|
<category>RPC</category>
|
|
<category>SPI</category>
|
|
<category>Dubbo</category>
|
|
<category>SPI</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Dubbo</tag>
|
|
<tag>RPC</tag>
|
|
<tag>SPI</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 Dubbo 的容错机制</title>
|
|
<url>/2020/11/22/%E8%81%8A%E8%81%8A-Dubbo-%E7%9A%84%E5%AE%B9%E9%94%99%E6%9C%BA%E5%88%B6/</url>
|
|
<content><![CDATA[<p>之前看了 dubbo 的一些代码，在学习过程中，主要关注那些比较“高级”的内容，SPI，自适应扩展等，却忘了一些作为一个 rpc 框架最核心需要的部分，比如如何通信，序列化，网络，容错机制等等，因为其实这个最核心的就是远程调用，自适应扩展其实就是让代码可扩展性，可读性，更优雅等，写的搓一点其实也问题不大，但是一个合适的通信协议，序列化方法，如何容错等却是真正保证是一个 rpc 框架最重要的要素。<br>首先来看这张图<br><img data-src="https://img.nicksxs.me/uPic/cluster.jpg" alt="cluster"><br>在集群调用失败时，Dubbo 提供了多种容错方案，缺省为 failover 重试。<br>各节点关系：</p>
|
|
<ul>
|
|
<li>这里的 <code>Invoker</code> 是 <code>Provider</code> 的一个可调用 <code>Service</code> 的抽象，<code>Invoker</code> 封装了 <code>Provider</code> 地址及 <code>Service</code> 接口信息</li>
|
|
<li><code>Directory</code> 代表多个 <code>Invoker</code>，可以把它看成 <code>List<Invoker></code> ，但与 <code>List</code> 不同的是，它的值可能是动态变化的，比如注册中心推送变更</li>
|
|
<li><code>Cluster</code> 将 <code>Directory</code> 中的多个 <code>Invoker</code> 伪装成一个 <code>Invoker</code>，对上层透明，伪装过程包含了容错逻辑，调用失败后，重试另一个</li>
|
|
<li><code>Router</code> 负责从多个 <code>Invoker</code> 中按路由规则选出子集，比如读写分离，应用隔离等</li>
|
|
<li><code>LoadBalance</code> 负责从多个 <code>Invoker</code> 中选出具体的一个用于本次调用，选的过程包含了负载均衡算法，调用失败后，需要重选</li>
|
|
</ul>
|
|
<h2 id="集群容错模式"><a href="#集群容错模式" class="headerlink" title="集群容错模式"></a>集群容错模式</h2><h3 id="Failover-Cluster"><a href="#Failover-Cluster" class="headerlink" title="Failover Cluster"></a>Failover Cluster</h3><p>失败自动切换，当出现失败，重试其它服务器 1。通常用于读操作，但重试会带来更长延迟。可通过 retries=”2” 来设置重试次数(不含第一次)。</p>
|
|
<p>重试次数配置如下：</p>
|
|
<p><dubbo:service retries=”2” /><br>这里重点看下 <code>Failover Cluster</code>集群模式的实现 </p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">FailoverCluster</span> <span class="keyword">implements</span> <span class="title class_">Cluster</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">final</span> <span class="keyword">static</span> <span class="type">String</span> <span class="variable">NAME</span> <span class="operator">=</span> <span class="string">"failover"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <T> Invoker<T> <span class="title function_">join</span><span class="params">(Directory<T> directory)</span> <span class="keyword">throws</span> RpcException {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">FailoverClusterInvoker</span><T>(directory);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这个代码就非常简单，重点需要看<code>FailoverClusterInvoker</code>里的代码，<code>FailoverClusterInvoker</code>继承了<code>AbstractClusterInvoker</code>类，其中invoke 方法是在抽象类里实现的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> Result <span class="title function_">invoke</span><span class="params">(<span class="keyword">final</span> Invocation invocation)</span> <span class="keyword">throws</span> RpcException {</span><br><span class="line"> checkWhetherDestroyed();</span><br><span class="line"> <span class="comment">// binding attachments into invocation.</span></span><br><span class="line"> <span class="comment">// 绑定 attachments 到 invocation 中.</span></span><br><span class="line"> Map<String, Object> contextAttachments = RpcContext.getContext().getObjectAttachments();</span><br><span class="line"> <span class="keyword">if</span> (contextAttachments != <span class="literal">null</span> && contextAttachments.size() != <span class="number">0</span>) {</span><br><span class="line"> ((RpcInvocation) invocation).addObjectAttachments(contextAttachments);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 列举 Invoker</span></span><br><span class="line"> List<Invoker<T>> invokers = list(invocation);</span><br><span class="line"> <span class="comment">// 加载 LoadBalance 负载均衡器</span></span><br><span class="line"> <span class="type">LoadBalance</span> <span class="variable">loadbalance</span> <span class="operator">=</span> initLoadBalance(invokers, invocation);</span><br><span class="line"> RpcUtils.attachInvocationIdIfAsync(getUrl(), invocation);</span><br><span class="line"> <span class="comment">// 调用 实际的 doInvoke 进行后续操作</span></span><br><span class="line"> <span class="keyword">return</span> doInvoke(invocation, invokers, loadbalance);</span><br><span class="line">}</span><br><span class="line"><span class="comment">// 这是个抽象方法，实际是由子类实现的</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="keyword">abstract</span> Result <span class="title function_">doInvoke</span><span class="params">(Invocation invocation, List<Invoker<T>> invokers,</span></span><br><span class="line"><span class="params"> LoadBalance loadbalance)</span> <span class="keyword">throws</span> RpcException;</span><br></pre></td></tr></table></figure>
|
|
<p>然后重点就是<code>FailoverClusterInvoker</code>中的<code>doInvoke</code>方法了，其实它里面也就这么一个方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="meta">@SuppressWarnings({"unchecked", "rawtypes"})</span></span><br><span class="line"> <span class="keyword">public</span> Result <span class="title function_">doInvoke</span><span class="params">(Invocation invocation, <span class="keyword">final</span> List<Invoker<T>> invokers, LoadBalance loadbalance)</span> <span class="keyword">throws</span> RpcException {</span><br><span class="line"> List<Invoker<T>> copyInvokers = invokers;</span><br><span class="line"> checkInvokers(copyInvokers, invocation);</span><br><span class="line"> <span class="type">String</span> <span class="variable">methodName</span> <span class="operator">=</span> RpcUtils.getMethodName(invocation);</span><br><span class="line"> <span class="comment">// 获取重试次数，这里默认是 2 次，还有可以注意下后面的+1</span></span><br><span class="line"> <span class="type">int</span> <span class="variable">len</span> <span class="operator">=</span> getUrl().getMethodParameter(methodName, RETRIES_KEY, DEFAULT_RETRIES) + <span class="number">1</span>;</span><br><span class="line"> <span class="keyword">if</span> (len <= <span class="number">0</span>) {</span><br><span class="line"> len = <span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// retry loop.</span></span><br><span class="line"> <span class="type">RpcException</span> <span class="variable">le</span> <span class="operator">=</span> <span class="literal">null</span>; <span class="comment">// last exception.</span></span><br><span class="line"> List<Invoker<T>> invoked = <span class="keyword">new</span> <span class="title class_">ArrayList</span><Invoker<T>>(copyInvokers.size()); <span class="comment">// invoked invokers.</span></span><br><span class="line"> Set<String> providers = <span class="keyword">new</span> <span class="title class_">HashSet</span><String>(len);</span><br><span class="line"> <span class="comment">// 循环调用，失败重试</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i < len; i++) {</span><br><span class="line"> <span class="comment">//Reselect before retry to avoid a change of candidate `invokers`.</span></span><br><span class="line"> <span class="comment">//<span class="doctag">NOTE:</span> if `invokers` changed, then `invoked` also lose accuracy.</span></span><br><span class="line"> <span class="keyword">if</span> (i > <span class="number">0</span>) {</span><br><span class="line"> checkWhetherDestroyed();</span><br><span class="line"> <span class="comment">// 在进行重试前重新列举 Invoker，这样做的好处是，如果某个服务挂了，</span></span><br><span class="line"> <span class="comment">// 通过调用 list 可得到最新可用的 Invoker 列表</span></span><br><span class="line"> copyInvokers = list(invocation);</span><br><span class="line"> <span class="comment">// check again</span></span><br><span class="line"> <span class="comment">// 对 copyinvokers 进行判空检查</span></span><br><span class="line"> checkInvokers(copyInvokers, invocation);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 通过负载均衡来选择 invoker</span></span><br><span class="line"> Invoker<T> invoker = select(loadbalance, invocation, copyInvokers, invoked);</span><br><span class="line"> <span class="comment">// 将其添加到 invoker 到 invoked 列表中</span></span><br><span class="line"> invoked.add(invoker);</span><br><span class="line"> <span class="comment">// 设置上下文</span></span><br><span class="line"> RpcContext.getContext().setInvokers((List) invoked);</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 正式调用</span></span><br><span class="line"> <span class="type">Result</span> <span class="variable">result</span> <span class="operator">=</span> invoker.invoke(invocation);</span><br><span class="line"> <span class="keyword">if</span> (le != <span class="literal">null</span> && logger.isWarnEnabled()) {</span><br><span class="line"> logger.warn(<span class="string">"Although retry the method "</span> + methodName</span><br><span class="line"> + <span class="string">" in the service "</span> + getInterface().getName()</span><br><span class="line"> + <span class="string">" was successful by the provider "</span> + invoker.getUrl().getAddress()</span><br><span class="line"> + <span class="string">", but there have been failed providers "</span> + providers</span><br><span class="line"> + <span class="string">" ("</span> + providers.size() + <span class="string">"/"</span> + copyInvokers.size()</span><br><span class="line"> + <span class="string">") from the registry "</span> + directory.getUrl().getAddress()</span><br><span class="line"> + <span class="string">" on the consumer "</span> + NetUtils.getLocalHost()</span><br><span class="line"> + <span class="string">" using the dubbo version "</span> + Version.getVersion() + <span class="string">". Last error is: "</span></span><br><span class="line"> + le.getMessage(), le);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> result;</span><br><span class="line"> } <span class="keyword">catch</span> (RpcException e) {</span><br><span class="line"> <span class="keyword">if</span> (e.isBiz()) { <span class="comment">// biz exception.</span></span><br><span class="line"> <span class="keyword">throw</span> e;</span><br><span class="line"> }</span><br><span class="line"> le = e;</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> le = <span class="keyword">new</span> <span class="title class_">RpcException</span>(e.getMessage(), e);</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> providers.add(invoker.getUrl().getAddress());</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">RpcException</span>(le.getCode(), <span class="string">"Failed to invoke the method "</span></span><br><span class="line"> + methodName + <span class="string">" in the service "</span> + getInterface().getName()</span><br><span class="line"> + <span class="string">". Tried "</span> + len + <span class="string">" times of the providers "</span> + providers</span><br><span class="line"> + <span class="string">" ("</span> + providers.size() + <span class="string">"/"</span> + copyInvokers.size()</span><br><span class="line"> + <span class="string">") from the registry "</span> + directory.getUrl().getAddress()</span><br><span class="line"> + <span class="string">" on the consumer "</span> + NetUtils.getLocalHost() + <span class="string">" using the dubbo version "</span></span><br><span class="line"> + Version.getVersion() + <span class="string">". Last error is: "</span></span><br><span class="line"> + le.getMessage(), le.getCause() != <span class="literal">null</span> ? le.getCause() : le);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
<h3 id="Failfast-Cluster"><a href="#Failfast-Cluster" class="headerlink" title="Failfast Cluster"></a>Failfast Cluster</h3><p>快速失败，只发起一次调用，失败立即报错。通常用于非幂等性的写操作，比如新增记录。</p>
|
|
<h3 id="Failsafe-Cluster"><a href="#Failsafe-Cluster" class="headerlink" title="Failsafe Cluster"></a>Failsafe Cluster</h3><p>失败安全，出现异常时，直接忽略。通常用于写入审计日志等操作。</p>
|
|
<h3 id="Failback-Cluster"><a href="#Failback-Cluster" class="headerlink" title="Failback Cluster"></a>Failback Cluster</h3><p>失败自动恢复，后台记录失败请求，定时重发。通常用于消息通知操作。</p>
|
|
<h3 id="Forking-Cluster"><a href="#Forking-Cluster" class="headerlink" title="Forking Cluster"></a>Forking Cluster</h3><p>并行调用多个服务器，只要一个成功即返回。通常用于实时性要求较高的读操作，但需要浪费更多服务资源。可通过 forks=”2” 来设置最大并行数。</p>
|
|
<h3 id="Broadcast-Cluster"><a href="#Broadcast-Cluster" class="headerlink" title="Broadcast Cluster"></a>Broadcast Cluster</h3><p>广播调用所有提供者，逐个调用，任意一台报错则报错 2。通常用于通知所有提供者更新缓存或日志等本地资源信息。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Dubbo - RPC</category>
|
|
<category>Dubbo</category>
|
|
<category>容错机制</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Dubbo</tag>
|
|
<tag>RPC</tag>
|
|
<tag>容错机制</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 Java 中绕不开的 Synchronized 关键字-二</title>
|
|
<url>/2021/06/27/%E8%81%8A%E8%81%8A-Java-%E4%B8%AD%E7%BB%95%E4%B8%8D%E5%BC%80%E7%9A%84-Synchronized-%E5%85%B3%E9%94%AE%E5%AD%97-%E4%BA%8C/</url>
|
|
<content><![CDATA[<h1 id="Java并发"><a href="#Java并发" class="headerlink" title="Java并发"></a>Java并发</h1><p>synchronized 的一些学习记录</p>
|
|
<p>jdk1.6 以后对 synchronized 进行了一些优化，包括偏向锁，轻量级锁，重量级锁等</p>
|
|
<p>这些锁的加锁方式大多跟对象头有关，我们可以查看 jdk 代码</p>
|
|
<p>首先对象头的位置注释</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="comment">// Bit-format of an object header (most significant first, big endian layout below):</span></span><br><span class="line"><span class="comment">//</span></span><br><span class="line"><span class="comment">// 32 bits:</span></span><br><span class="line"><span class="comment">// --------</span></span><br><span class="line"><span class="comment">// hash:25 ------------>| age:4 biased_lock:1 lock:2 (normal object)</span></span><br><span class="line"><span class="comment">// JavaThread*:23 epoch:2 age:4 biased_lock:1 lock:2 (biased object)</span></span><br><span class="line"><span class="comment">// size:32 ------------------------------------------>| (CMS free block)</span></span><br><span class="line"><span class="comment">// PromotedObject*:29 ---------->| promo_bits:3 ----->| (CMS promoted object)</span></span><br><span class="line"><span class="comment">//</span></span><br><span class="line"><span class="comment">// 64 bits:</span></span><br><span class="line"><span class="comment">// --------</span></span><br><span class="line"><span class="comment">// unused:25 hash:31 -->| unused:1 age:4 biased_lock:1 lock:2 (normal object)</span></span><br><span class="line"><span class="comment">// JavaThread*:54 epoch:2 unused:1 age:4 biased_lock:1 lock:2 (biased object)</span></span><br><span class="line"><span class="comment">// PromotedObject*:61 --------------------->| promo_bits:3 ----->| (CMS promoted object)</span></span><br><span class="line"><span class="comment">// size:64 ----------------------------------------------------->| (CMS free block)</span></span><br><span class="line"><span class="comment">//</span></span><br><span class="line"><span class="comment">// unused:25 hash:31 -->| cms_free:1 age:4 biased_lock:1 lock:2 (COOPs && normal object)</span></span><br><span class="line"><span class="comment">// JavaThread*:54 epoch:2 cms_free:1 age:4 biased_lock:1 lock:2 (COOPs && biased object)</span></span><br><span class="line"><span class="comment">// narrowOop:32 unused:24 cms_free:1 unused:4 promo_bits:3 ----->| (COOPs && CMS promoted object)</span></span><br><span class="line"><span class="comment">// unused:21 size:35 -->| cms_free:1 unused:7 ------------------>| (COOPs && CMS free block)</span></span><br></pre></td></tr></table></figure>
|
|
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="class"><span class="keyword">enum</span> {</span> locked_value = <span class="number">0</span>,</span><br><span class="line"> unlocked_value = <span class="number">1</span>,</span><br><span class="line"> monitor_value = <span class="number">2</span>,</span><br><span class="line"> marked_value = <span class="number">3</span>,</span><br><span class="line"> biased_lock_pattern = <span class="number">5</span></span><br><span class="line">};</span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
|
|
<p>我们可以用 java jol库来查看对象头，通过一段简单的代码来看下</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">ObjectHeaderDemo</span> {</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> <span class="keyword">throws</span> InterruptedException {</span><br><span class="line"> <span class="type">L</span> <span class="variable">l</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">L</span>();</span><br><span class="line"> System.out.println(ClassLayout.parseInstance(l).toPrintable());</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p><img data-src="https://img.nicksxs.me/uPic/Untitled.png" alt="Untitled"></p>
|
|
<p>然后可以看到打印输出，当然这里因为对齐方式，我们看到的其实顺序是反过来的，按最后三位去看，我们这是 001，好像偏向锁都没开，这里使用的是 jdk1.8，默认开始偏向锁的，其实这里有涉及到了一个配置，jdk1.8 中偏向锁会延迟 4 秒开启，可以通过添加启动参数 -XX:+PrintFlagsFinal，看到</p>
|
|
<p><img data-src="https://img.nicksxs.me/uPic/%E5%81%8F%E5%90%91%E9%94%81%E5%BB%B6%E8%BF%9F.png" alt="偏向锁延迟"></p>
|
|
<p>因为在初始化的时候防止线程竞争有大量的偏向锁撤销升级，所以会延迟 4s 开启</p>
|
|
<p>我们再来延迟 5s 看看</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">ObjectHeaderDemo</span> {</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> <span class="keyword">throws</span> InterruptedException {</span><br><span class="line"> TimeUnit.SECONDS.sleep(<span class="number">5</span>);</span><br><span class="line"> <span class="type">L</span> <span class="variable">l</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">L</span>();</span><br><span class="line"> System.out.println(ClassLayout.parseInstance(l).toPrintable());</span><br><span class="line"> }</span><br><span class="line">} </span><br></pre></td></tr></table></figure>
|
|
|
|
<p><img data-src="https://img.nicksxs.me/uPic/2LBKpX.jpg" alt="https://img.nicksxs.me/uPic/2LBKpX.jpg"></p>
|
|
<p>可以看到偏向锁设置已经开启了，我们来是一下加个偏向锁</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">ObjectHeaderDemo</span> {</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> <span class="keyword">throws</span> InterruptedException {</span><br><span class="line"> TimeUnit.SECONDS.sleep(<span class="number">5</span>);</span><br><span class="line"> <span class="type">L</span> <span class="variable">l</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">L</span>();</span><br><span class="line"> System.out.println(ClassLayout.parseInstance(l).toPrintable());</span><br><span class="line"> <span class="keyword">synchronized</span> (l) {</span><br><span class="line"> System.out.println(<span class="string">"1\n"</span> + ClassLayout.parseInstance(l).toPrintable());</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">synchronized</span> (l) {</span><br><span class="line"> System.out.println(<span class="string">"2\n"</span> + ClassLayout.parseInstance(l).toPrintable());</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>看下运行结果</p>
|
|
<p><img data-src="https://img.nicksxs.me/uPic/V2l78m.png" alt="https://img.nicksxs.me/uPic/V2l78m.png"></p>
|
|
<p>可以看到是加上了 101 = 5 也就是偏向锁，后面是线程 id</p>
|
|
<p>当我再使用一个线程来竞争这个锁的时候</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">ObjectHeaderDemo</span> {</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> <span class="keyword">throws</span> InterruptedException {</span><br><span class="line"> TimeUnit.SECONDS.sleep(<span class="number">5</span>);</span><br><span class="line"> <span class="type">L</span> <span class="variable">l</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">L</span>();</span><br><span class="line"> System.out.println(ClassLayout.parseInstance(l).toPrintable());</span><br><span class="line"> <span class="keyword">synchronized</span> (l) {</span><br><span class="line"> System.out.println(<span class="string">"1\n"</span> + ClassLayout.parseInstance(l).toPrintable());</span><br><span class="line"> }</span><br><span class="line"> <span class="type">Thread</span> <span class="variable">thread1</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Thread</span>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> TimeUnit.SECONDS.sleep(<span class="number">5L</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">synchronized</span> (l) {</span><br><span class="line"> System.out.println(<span class="string">"thread1 获取锁成功"</span>);</span><br><span class="line"> System.out.println(ClassLayout.parseInstance(l).toPrintable());</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> TimeUnit.SECONDS.sleep(<span class="number">5L</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> };</span><br><span class="line"> thread1.start();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p><img data-src="https://img.nicksxs.me/uPic/bRMvlR.png" alt="https://img.nicksxs.me/uPic/bRMvlR.png"></p>
|
|
<p>可以看到变成了轻量级锁，在线程没有争抢，只是进行了切换，就会使用轻量级锁，当两个线程在竞争了，就又会升级成重量级锁</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">ObjectHeaderDemo</span> {</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> <span class="keyword">throws</span> InterruptedException {</span><br><span class="line"> TimeUnit.SECONDS.sleep(<span class="number">5</span>);</span><br><span class="line"> <span class="type">L</span> <span class="variable">l</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">L</span>();</span><br><span class="line"> System.out.println(ClassLayout.parseInstance(l).toPrintable());</span><br><span class="line"> <span class="keyword">synchronized</span> (l) {</span><br><span class="line"> System.out.println(<span class="string">"1\n"</span> + ClassLayout.parseInstance(l).toPrintable());</span><br><span class="line"> }</span><br><span class="line"> <span class="type">Thread</span> <span class="variable">thread1</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Thread</span>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> TimeUnit.SECONDS.sleep(<span class="number">5L</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">synchronized</span> (l) {</span><br><span class="line"> System.out.println(<span class="string">"thread1 获取锁成功"</span>);</span><br><span class="line"> System.out.println(ClassLayout.parseInstance(l).toPrintable());</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> TimeUnit.SECONDS.sleep(<span class="number">5L</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> };</span><br><span class="line"> <span class="type">Thread</span> <span class="variable">thread2</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Thread</span>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> TimeUnit.SECONDS.sleep(<span class="number">5L</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">synchronized</span> (l) {</span><br><span class="line"> System.out.println(<span class="string">"thread2 获取锁成功"</span>);</span><br><span class="line"> System.out.println(ClassLayout.parseInstance(l).toPrintable());</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> };</span><br><span class="line"> thread1.start();</span><br><span class="line"> thread2.start();</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">class</span> <span class="title class_">L</span> {</span><br><span class="line"> <span class="keyword">private</span> <span class="type">boolean</span> <span class="variable">myboolean</span> <span class="operator">=</span> <span class="literal">true</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p><img data-src="https://img.nicksxs.me/uPic/LMzMtR.png" alt="https://img.nicksxs.me/uPic/LMzMtR.png"></p>
|
|
<p>可以看到变成了重量级锁。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Synchronized</tag>
|
|
<tag>偏向锁</tag>
|
|
<tag>轻量级锁</tag>
|
|
<tag>重量级锁</tag>
|
|
<tag>自旋</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 Dubbo 的 SPI 续之自适应拓展</title>
|
|
<url>/2020/06/06/%E8%81%8A%E8%81%8A-Dubbo-%E7%9A%84-SPI-%E7%BB%AD%E4%B9%8B%E8%87%AA%E9%80%82%E5%BA%94%E6%8B%93%E5%B1%95/</url>
|
|
<content><![CDATA[<h2 id="Adaptive"><a href="#Adaptive" class="headerlink" title="Adaptive"></a>Adaptive</h2><p>这个应该是 Dubbo SPI 里最玄妙的东西了，一开始没懂，自适应扩展点加载，<br><code>dubbo://123.123.123.123:1234/com.nicksxs.demo.service.HelloWorldService?anyhost=true&application=demo&default.loadbalance=random&default.service.filter=LoggerFilter&dubbo=2.5.3&interface=com.nicksxs.demo.service.HelloWorldService&logger=slf4j&methods=method1,method2,method3,method4&pid=4292&retries=0&side=provider&threadpool=fixed&threads=200&timeout=2000&timestamp=1590647155886</code><br>那我从比较能理解的角度或者说思路去讲讲我的理解，因为直接将原理如果脱离了使用，对于我这样的理解能力比较差的可能会比较吃力，从使用场景开始讲可能会比较舒服了，这里可以看到参数里有蛮多的，举个例子，比如这个 <code>threadpool = fixed</code>，说明线程池使用的是 fixed 对应的实现，也就是下图的这个<br><img data-src="https://mystore-1255223546.cos.ap-chengdu.myqcloud.com/uPic/4AAOaW.png"><br>这样子似乎没啥问题了，反正就是用dubbo 的 spi 加载嘛，好像没啥问题，其实问题还是存在的，或者说不太优雅，比如要先判断我这个 fixed 对应的实现类是哪个，这里可能就有个 if-else 判断了，但是 dubbo 的开发人员似乎不太想这么做这个事情，</p>
|
|
<p>譬如我们在引用一个服务时，在ReferenceConfig 中的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="type">Protocol</span> <span class="variable">refprotocol</span> <span class="operator">=</span> ExtensionLoader.getExtensionLoader(Protocol.class).getAdaptiveExtension();</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>就获取了自适应拓展，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> T <span class="title function_">getAdaptiveExtension</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">Object</span> <span class="variable">instance</span> <span class="operator">=</span> cachedAdaptiveInstance.get();</span><br><span class="line"> <span class="keyword">if</span> (instance == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">if</span> (createAdaptiveInstanceError == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">synchronized</span> (cachedAdaptiveInstance) {</span><br><span class="line"> instance = cachedAdaptiveInstance.get();</span><br><span class="line"> <span class="keyword">if</span> (instance == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> instance = createAdaptiveExtension();</span><br><span class="line"> cachedAdaptiveInstance.set(instance);</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable t) {</span><br><span class="line"> createAdaptiveInstanceError = t;</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalStateException</span>(<span class="string">"fail to create adaptive instance: "</span> + t.toString(), t);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalStateException</span>(<span class="string">"fail to create adaptive instance: "</span> + createAdaptiveInstanceError.toString(), createAdaptiveInstanceError);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> (T) instance;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这里也使用了 DCL，来锁cachedAdaptiveInstance，当缓存中没有时就去创建自适应拓展</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> T <span class="title function_">createAdaptiveExtension</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 获取自适应拓展类然后实例化</span></span><br><span class="line"> <span class="keyword">return</span> injectExtension((T) getAdaptiveExtensionClass().newInstance());</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalStateException</span>(<span class="string">"Can not create adaptive extension "</span> + type + <span class="string">", cause: "</span> + e.getMessage(), e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"><span class="keyword">private</span> Class<?> getAdaptiveExtensionClass() {</span><br><span class="line"> <span class="comment">// 这里会获取拓展类，如果没有自适应的拓展类，那么就需要调用createAdaptiveExtensionClass</span></span><br><span class="line"> getExtensionClasses();</span><br><span class="line"> <span class="keyword">if</span> (cachedAdaptiveClass != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> cachedAdaptiveClass;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">return</span> <span class="variable">cachedAdaptiveClass</span> <span class="operator">=</span> createAdaptiveExtensionClass();</span><br><span class="line"> }</span><br><span class="line"><span class="keyword">private</span> Class<?> createAdaptiveExtensionClass() {</span><br><span class="line"> <span class="comment">// 这里去生成了自适应拓展的代码，具体生成逻辑比较复杂先不展开讲</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">code</span> <span class="operator">=</span> createAdaptiveExtensionClassCode();</span><br><span class="line"> <span class="type">ClassLoader</span> <span class="variable">classLoader</span> <span class="operator">=</span> findClassLoader();</span><br><span class="line"> com.alibaba.dubbo.common.compiler.<span class="type">Compiler</span> <span class="variable">compiler</span> <span class="operator">=</span> ExtensionLoader.getExtensionLoader(com.alibaba.dubbo.common.compiler.Compiler.class).getAdaptiveExtension();</span><br><span class="line"> <span class="keyword">return</span> compiler.compile(code, classLoader);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>生成的代码像这样</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">package</span> com.alibaba.dubbo.rpc;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> com.alibaba.dubbo.common.extension.ExtensionLoader;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">Protocol$Adaptive</span> <span class="keyword">implements</span> <span class="title class_">com</span>.alibaba.dubbo.rpc.Protocol {</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">destroy</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">UnsupportedOperationException</span>(</span><br><span class="line"> <span class="string">"method public abstract void com.alibaba.dubbo.rpc.Protocol.destroy() of interface com.alibaba.dubbo.rpc.Protocol is not adaptive method!"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="type">int</span> <span class="title function_">getDefaultPort</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">UnsupportedOperationException</span>(</span><br><span class="line"> <span class="string">"method public abstract int com.alibaba.dubbo.rpc.Protocol.getDefaultPort() of interface com.alibaba.dubbo.rpc.Protocol is not adaptive method!"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> com.alibaba.dubbo.rpc.Exporter <span class="title function_">export</span><span class="params">(</span></span><br><span class="line"><span class="params"> com.alibaba.dubbo.rpc.Invoker arg0)</span></span><br><span class="line"> <span class="keyword">throws</span> com.alibaba.dubbo.rpc.RpcException {</span><br><span class="line"> <span class="keyword">if</span> (arg0 == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalArgumentException</span>(</span><br><span class="line"> <span class="string">"com.alibaba.dubbo.rpc.Invoker argument == null"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (arg0.getUrl() == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalArgumentException</span>(</span><br><span class="line"> <span class="string">"com.alibaba.dubbo.rpc.Invoker argument getUrl() == null"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> com.alibaba.dubbo.common.<span class="type">URL</span> <span class="variable">url</span> <span class="operator">=</span> arg0.getUrl();</span><br><span class="line"> <span class="type">String</span> <span class="variable">extName</span> <span class="operator">=</span> ((url.getProtocol() == <span class="literal">null</span>) ? <span class="string">"dubbo"</span></span><br><span class="line"> : url.getProtocol());</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (extName == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalStateException</span>(</span><br><span class="line"> <span class="string">"Fail to get extension(com.alibaba.dubbo.rpc.Protocol) name from url("</span> +</span><br><span class="line"> url.toString() + <span class="string">") use keys([protocol])"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> com.alibaba.dubbo.rpc.<span class="type">Protocol</span> <span class="variable">extension</span> <span class="operator">=</span> (com.alibaba.dubbo.rpc.Protocol) ExtensionLoader.getExtensionLoader(com.alibaba.dubbo.rpc.Protocol.class)</span><br><span class="line"> .getExtension(extName);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> extension.export(arg0);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> com.alibaba.dubbo.rpc.Invoker <span class="title function_">refer</span><span class="params">(java.lang.Class arg0,</span></span><br><span class="line"><span class="params"> com.alibaba.dubbo.common.URL arg1)</span></span><br><span class="line"> <span class="keyword">throws</span> com.alibaba.dubbo.rpc.RpcException {</span><br><span class="line"> <span class="keyword">if</span> (arg1 == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalArgumentException</span>(<span class="string">"url == null"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> com.alibaba.dubbo.common.<span class="type">URL</span> <span class="variable">url</span> <span class="operator">=</span> arg1;</span><br><span class="line"> <span class="comment">// 其实前面所说的逻辑就在这里呈现了</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">extName</span> <span class="operator">=</span> ((url.getProtocol() == <span class="literal">null</span>) ? <span class="string">"dubbo"</span></span><br><span class="line"> : url.getProtocol());</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (extName == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalStateException</span>(</span><br><span class="line"> <span class="string">"Fail to get extension(com.alibaba.dubbo.rpc.Protocol) name from url("</span> +</span><br><span class="line"> url.toString() + <span class="string">") use keys([protocol])"</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 在这就是实际的通过dubbo 的 spi 去加载实际对应的扩展</span></span><br><span class="line"> com.alibaba.dubbo.rpc.<span class="type">Protocol</span> <span class="variable">extension</span> <span class="operator">=</span> (com.alibaba.dubbo.rpc.Protocol) ExtensionLoader.getExtensionLoader(com.alibaba.dubbo.rpc.Protocol.class)</span><br><span class="line"> .getExtension(extName);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> extension.refer(arg0, arg1);</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Dubbo</category>
|
|
<category>RPC</category>
|
|
<category>SPI</category>
|
|
<category>Dubbo</category>
|
|
<category>SPI</category>
|
|
<category>Adaptive</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Dubbo</tag>
|
|
<tag>RPC</tag>
|
|
<tag>SPI</tag>
|
|
<tag>Adaptive</tag>
|
|
<tag>自适应拓展</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 Java 中绕不开的 Synchronized 关键字</title>
|
|
<url>/2021/06/20/%E8%81%8A%E8%81%8A-Java-%E4%B8%AD%E7%BB%95%E4%B8%8D%E5%BC%80%E7%9A%84-Synchronized-%E5%85%B3%E9%94%AE%E5%AD%97/</url>
|
|
<content><![CDATA[<p>Synchronized 关键字在 Java 的并发体系里也是非常重要的一个内容，首先比较常规的是知道它使用的方式，可以锁对象，可以锁代码块，也可以锁方法，看一个简单的 demo</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">SynchronizedDemo</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> <span class="type">SynchronizedDemo</span> <span class="variable">synchronizedDemo</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">SynchronizedDemo</span>();</span><br><span class="line"> synchronizedDemo.lockMethod();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title function_">lockMethod</span><span class="params">()</span> {</span><br><span class="line"> System.out.println(<span class="string">"here i'm locked"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">lockSynchronizedDemo</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">synchronized</span> (<span class="built_in">this</span>) {</span><br><span class="line"> System.out.println(<span class="string">"here lock class"</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>然后来查看反编译结果，其实代码（日光）之下并无新事，即使是完全不懂的也可以通过一些词义看出一些意义</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"> Last modified <span class="number">2021</span>年<span class="number">6</span>月<span class="number">20</span>日; size <span class="number">729</span> bytes</span><br><span class="line"> MD5 checksum dd9c529863bd7ff839a95481db578ad9</span><br><span class="line"> Compiled from <span class="string">"SynchronizedDemo.java"</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">SynchronizedDemo</span></span><br><span class="line"> minor version: <span class="number">0</span></span><br><span class="line"> major version: <span class="number">53</span></span><br><span class="line"> flags: (<span class="number">0x0021</span>) ACC_PUBLIC, ACC_SUPER</span><br><span class="line"> this_class: #<span class="number">2</span> <span class="comment">// SynchronizedDemo</span></span><br><span class="line"> super_class: #<span class="number">9</span> <span class="comment">// java/lang/Object</span></span><br><span class="line"> interfaces: <span class="number">0</span>, fields: <span class="number">0</span>, methods: <span class="number">4</span>, attributes: <span class="number">1</span></span><br><span class="line">Constant pool:</span><br><span class="line"> #<span class="number">1</span> = Methodref #<span class="number">9.</span>#<span class="number">22</span> <span class="comment">// java/lang/Object."<init>":()V</span></span><br><span class="line"> #<span class="number">2</span> = Class #<span class="number">23</span> <span class="comment">// SynchronizedDemo</span></span><br><span class="line"> #<span class="number">3</span> = Methodref #<span class="number">2.</span>#<span class="number">22</span> <span class="comment">// SynchronizedDemo."<init>":()V</span></span><br><span class="line"> #<span class="number">4</span> = Methodref #<span class="number">2.</span>#<span class="number">24</span> <span class="comment">// SynchronizedDemo.lockMethod:()V</span></span><br><span class="line"> #<span class="number">5</span> = Fieldref #<span class="number">25.</span>#<span class="number">26</span> <span class="comment">// java/lang/System.out:Ljava/io/PrintStream;</span></span><br><span class="line"> #<span class="number">6</span> = String #<span class="number">27</span> <span class="comment">// here i\'m locked</span></span><br><span class="line"> #<span class="number">7</span> = Methodref #<span class="number">28.</span>#<span class="number">29</span> <span class="comment">// java/io/PrintStream.println:(Ljava/lang/String;)V</span></span><br><span class="line"> #<span class="number">8</span> = String #<span class="number">30</span> <span class="comment">// here lock class</span></span><br><span class="line"> #<span class="number">9</span> = Class #<span class="number">31</span> <span class="comment">// java/lang/Object</span></span><br><span class="line"> #<span class="number">10</span> = Utf8 <init></span><br><span class="line"> #<span class="number">11</span> = Utf8 ()V</span><br><span class="line"> #<span class="number">12</span> = Utf8 Code</span><br><span class="line"> #<span class="number">13</span> = Utf8 LineNumberTable</span><br><span class="line"> #<span class="number">14</span> = Utf8 main</span><br><span class="line"> #<span class="number">15</span> = Utf8 ([Ljava/lang/String;)V</span><br><span class="line"> #<span class="number">16</span> = Utf8 lockMethod</span><br><span class="line"> #<span class="number">17</span> = Utf8 lockSynchronizedDemo</span><br><span class="line"> #<span class="number">18</span> = Utf8 StackMapTable</span><br><span class="line"> #<span class="number">19</span> = Class #<span class="number">32</span> <span class="comment">// java/lang/Throwable</span></span><br><span class="line"> #<span class="number">20</span> = Utf8 SourceFile</span><br><span class="line"> #<span class="number">21</span> = Utf8 SynchronizedDemo.java</span><br><span class="line"> #<span class="number">22</span> = NameAndType #<span class="number">10</span>:#<span class="number">11</span> <span class="comment">// "<init>":()V</span></span><br><span class="line"> #<span class="number">23</span> = Utf8 SynchronizedDemo</span><br><span class="line"> #<span class="number">24</span> = NameAndType #<span class="number">16</span>:#<span class="number">11</span> <span class="comment">// lockMethod:()V</span></span><br><span class="line"> #<span class="number">25</span> = Class #<span class="number">33</span> <span class="comment">// java/lang/System</span></span><br><span class="line"> #<span class="number">26</span> = NameAndType #<span class="number">34</span>:#<span class="number">35</span> <span class="comment">// out:Ljava/io/PrintStream;</span></span><br><span class="line"> #<span class="number">27</span> = Utf8 here i\<span class="string">'m locked</span></span><br><span class="line"><span class="string"> #28 = Class #36 // java/io/PrintStream</span></span><br><span class="line"><span class="string"> #29 = NameAndType #37:#38 // println:(Ljava/lang/String;)V</span></span><br><span class="line"><span class="string"> #30 = Utf8 here lock class</span></span><br><span class="line"><span class="string"> #31 = Utf8 java/lang/Object</span></span><br><span class="line"><span class="string"> #32 = Utf8 java/lang/Throwable</span></span><br><span class="line"><span class="string"> #33 = Utf8 java/lang/System</span></span><br><span class="line"><span class="string"> #34 = Utf8 out</span></span><br><span class="line"><span class="string"> #35 = Utf8 Ljava/io/PrintStream;</span></span><br><span class="line"><span class="string"> #36 = Utf8 java/io/PrintStream</span></span><br><span class="line"><span class="string"> #37 = Utf8 println</span></span><br><span class="line"><span class="string"> #38 = Utf8 (Ljava/lang/String;)V</span></span><br><span class="line"><span class="string">{</span></span><br><span class="line"><span class="string"> public SynchronizedDemo();</span></span><br><span class="line"><span class="string"> descriptor: ()V</span></span><br><span class="line"><span class="string"> flags: (0x0001) ACC_PUBLIC</span></span><br><span class="line"><span class="string"> Code:</span></span><br><span class="line"><span class="string"> stack=1, locals=1, args_size=1</span></span><br><span class="line"><span class="string"> 0: aload_0</span></span><br><span class="line"><span class="string"> 1: invokespecial #1 // Method java/lang/Object."<init>":()V</span></span><br><span class="line"><span class="string"> 4: return</span></span><br><span class="line"><span class="string"> LineNumberTable:</span></span><br><span class="line"><span class="string"> line 5: 0</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> public static void main(java.lang.String[]);</span></span><br><span class="line"><span class="string"> descriptor: ([Ljava/lang/String;)V</span></span><br><span class="line"><span class="string"> flags: (0x0009) ACC_PUBLIC, ACC_STATIC</span></span><br><span class="line"><span class="string"> Code:</span></span><br><span class="line"><span class="string"> stack=2, locals=2, args_size=1</span></span><br><span class="line"><span class="string"> 0: new #2 // class SynchronizedDemo</span></span><br><span class="line"><span class="string"> 3: dup</span></span><br><span class="line"><span class="string"> 4: invokespecial #3 // Method "<init>":()V</span></span><br><span class="line"><span class="string"> 7: astore_1</span></span><br><span class="line"><span class="string"> 8: aload_1</span></span><br><span class="line"><span class="string"> 9: invokevirtual #4 // Method lockMethod:()V</span></span><br><span class="line"><span class="string"> 12: return</span></span><br><span class="line"><span class="string"> LineNumberTable:</span></span><br><span class="line"><span class="string"> line 8: 0</span></span><br><span class="line"><span class="string"> line 9: 8</span></span><br><span class="line"><span class="string"> line 10: 12</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> public synchronized void lockMethod();</span></span><br><span class="line"><span class="string"> descriptor: ()V</span></span><br><span class="line"><span class="string"> flags: (0x0021) ACC_PUBLIC, ACC_SYNCHRONIZED</span></span><br><span class="line"><span class="string"> Code:</span></span><br><span class="line"><span class="string"> stack=2, locals=1, args_size=1</span></span><br><span class="line"><span class="string"> 0: getstatic #5 // Field java/lang/System.out:Ljava/io/PrintStream;</span></span><br><span class="line"><span class="string"> 3: ldc #6 // String here i\'m locked</span></span><br><span class="line"><span class="string"> 5: invokevirtual #7 // Method java/io/PrintStream.println:(Ljava/lang/String;)V</span></span><br><span class="line"><span class="string"> 8: return</span></span><br><span class="line"><span class="string"> LineNumberTable:</span></span><br><span class="line"><span class="string"> line 13: 0</span></span><br><span class="line"><span class="string"> line 14: 8</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> public void lockSynchronizedDemo();</span></span><br><span class="line"><span class="string"> descriptor: ()V</span></span><br><span class="line"><span class="string"> flags: (0x0001) ACC_PUBLIC</span></span><br><span class="line"><span class="string"> Code:</span></span><br><span class="line"><span class="string"> stack=2, locals=3, args_size=1</span></span><br><span class="line"><span class="string"> 0: aload_0</span></span><br><span class="line"><span class="string"> 1: dup</span></span><br><span class="line"><span class="string"> 2: astore_1</span></span><br><span class="line"><span class="string"> 3: monitorenter</span></span><br><span class="line"><span class="string"> 4: getstatic #5 // Field java/lang/System.out:Ljava/io/PrintStream;</span></span><br><span class="line"><span class="string"> 7: ldc #8 // String here lock class</span></span><br><span class="line"><span class="string"> 9: invokevirtual #7 // Method java/io/PrintStream.println:(Ljava/lang/String;)V</span></span><br><span class="line"><span class="string"> 12: aload_1</span></span><br><span class="line"><span class="string"> 13: monitorexit</span></span><br><span class="line"><span class="string"> 14: goto 22</span></span><br><span class="line"><span class="string"> 17: astore_2</span></span><br><span class="line"><span class="string"> 18: aload_1</span></span><br><span class="line"><span class="string"> 19: monitorexit</span></span><br><span class="line"><span class="string"> 20: aload_2</span></span><br><span class="line"><span class="string"> 21: athrow</span></span><br><span class="line"><span class="string"> 22: return</span></span><br><span class="line"><span class="string"> Exception table:</span></span><br><span class="line"><span class="string"> from to target type</span></span><br><span class="line"><span class="string"> 4 14 17 any</span></span><br><span class="line"><span class="string"> 17 20 17 any</span></span><br><span class="line"><span class="string"> LineNumberTable:</span></span><br><span class="line"><span class="string"> line 17: 0</span></span><br><span class="line"><span class="string"> line 18: 4</span></span><br><span class="line"><span class="string"> line 19: 12</span></span><br><span class="line"><span class="string"> line 20: 22</span></span><br><span class="line"><span class="string"> StackMapTable: number_of_entries = 2</span></span><br><span class="line"><span class="string"> frame_type = 255 /* full_frame */</span></span><br><span class="line"><span class="string"> offset_delta = 17</span></span><br><span class="line"><span class="string"> locals = [ class SynchronizedDemo, class java/lang/Object ]</span></span><br><span class="line"><span class="string"> stack = [ class java/lang/Throwable ]</span></span><br><span class="line"><span class="string"> frame_type = 250 /* chop */</span></span><br><span class="line"><span class="string"> offset_delta = 4</span></span><br><span class="line"><span class="string">}</span></span><br><span class="line"><span class="string">SourceFile: "SynchronizedDemo.java"</span></span><br></pre></td></tr></table></figure>
|
|
|
|
<p>其中<code>lockMethod</code>中可以看到是通过 <code>ACC_SYNCHRONIZED</code> flag 来标记是被 synchronized 修饰，前面的 ACC 应该是 access 的意思，并且通过 <code>ACC_PUBLIC</code> 也可以看出来他们是同一类访问权限关键字来控制的，而修饰类则是通过<code>3: monitorenter</code>和<code>13: monitorexit</code>来控制并发，这个是原来就知道，后来看了下才知道修饰方法是不一样的，但是在前期都比较诟病是 synchronized 的性能，像 monitor 也是通过操作系统的<code>mutex lock</code>互斥锁来实现的，相对是比较重的锁，于是在 JDK 1.6 之后对 synchronized 做了一系列优化，包括偏向锁，轻量级锁，并且包括像 ConcurrentHashMap 这类并发集合都有在使用 synchronized 关键字配合 cas 来做并发保护，</p>
|
|
<p>jdk 对于 synchronized 的优化主要在于多重状态锁的升级，最初会使用偏向锁，当一个线程访问同步块并获取锁时，会在对象头和栈帧中的锁记录里存储锁偏向的线程ID，以后该线程在进入和退出同步块时不需要进行CAS操作来加锁和解锁，只需简单地测试一下对象头的Mark Word里是否存储着指向当前线程的偏向锁。引入偏向锁是为了在无多线程竞争的情况下尽量减少不必要的轻量级锁执行路径，因为轻量级锁的获取及释放依赖多次CAS原子指令，而偏向锁只需要在置换ThreadID的时候依赖一次CAS原子指令（由于一旦出现多线程竞争的情况就必须撤销偏向锁，所以偏向锁的撤销操作的性能损耗必须小于节省下来的CAS原子指令的性能消耗）。<br>而当出现线程尝试进入同步块时发现已有偏向锁，并且是其他线程时，会将锁升级成轻量级锁，并且自旋尝试获取锁，如果自旋成功则表示获取轻量级锁成功，否则将会升级成重量级锁进行阻塞，当然这里具体的还很复杂，说的比较浅薄主体还是想将原先的阻塞互斥锁进行轻量化，区分特殊情况进行加锁。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Synchronized</tag>
|
|
<tag>偏向锁</tag>
|
|
<tag>轻量级锁</tag>
|
|
<tag>重量级锁</tag>
|
|
<tag>自旋</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 Java 的类加载机制一</title>
|
|
<url>/2020/11/08/%E8%81%8A%E8%81%8A-Java-%E7%9A%84%E7%B1%BB%E5%8A%A0%E8%BD%BD%E6%9C%BA%E5%88%B6/</url>
|
|
<content><![CDATA[<p>一说到这个主题，想到的应该是双亲委派模型，不过讲的包括但不限于这个，主要内容是参考深入理解 Java 虚拟机书中的介绍，<br>一个类型的生命周期包含了七个阶段，加载，验证，准备，解析，初始化，使用，卸载。</p>
|
|
<ul>
|
|
<li><h2 id="加载"><a href="#加载" class="headerlink" title="加载"></a>加载</h2></li>
|
|
</ul>
|
|
<ol>
|
|
<li>通过一个类的全限定名来获取定义此类的二进制字节流</li>
|
|
<li>将这个字节流代表的静态存储结构转化为方法区的运行时数据结构</li>
|
|
<li>在内存中生成了一个代表这个类的 java.lang.Class 对象，作为方法区这个类的各种数据的访问入口</li>
|
|
</ol>
|
|
<ul>
|
|
<li><h2 id="验证"><a href="#验证" class="headerlink" title="验证"></a>验证</h2></li>
|
|
</ul>
|
|
<ol>
|
|
<li>文件格式验证</li>
|
|
<li>元数据验证</li>
|
|
<li>字节码验证</li>
|
|
<li>符号引用验证</li>
|
|
</ol>
|
|
<ul>
|
|
<li><h2 id="准备"><a href="#准备" class="headerlink" title="准备"></a>准备</h2></li>
|
|
</ul>
|
|
<p>准备阶段是正式为类中定义的变量（即静态变量，被static修饰的变量）分配内存并设置类变量初始值的阶段</p>
|
|
<ul>
|
|
<li><h2 id="解析"><a href="#解析" class="headerlink" title="解析"></a>解析</h2></li>
|
|
</ul>
|
|
<p>解析阶段是 Java 虚拟机将常量池内的符号引用替换为直接引用的过程</p>
|
|
<p>以上<a href="##%E9%93%BE%E6%8E%A5">验证</a>、<a href="##%E5%87%86%E5%A4%87">准备</a>、<a href="##%E8%A7%A3%E6%9E%90">解析</a> 三个阶段又合称为链接阶段，链接阶段要做的是将加载到JVM中的二进制字节流的类数据信息合并到JVM的运行时状态中。</p>
|
|
<ul>
|
|
<li><h2 id="初始化"><a href="#初始化" class="headerlink" title="初始化"></a>初始化</h2></li>
|
|
</ul>
|
|
<p>类的初始化阶段是类加载过程的最后一个步骤，也是除了自定义类加载器之外将主动权交给了应用程序，其实就是执行类构造器<clinit>()方法的过程，<clinit>()并不是我们在 Java 代码中直接编写的方法，它是 Javac编译器的自动生成物，<clinit>()方法是由编译器自动收集类中的所有类变量的复制动作和静态句块(static{}块)中的语句合并产生的，编译器收集的顺序是由语句在原文件中出现的顺序决定的，静态语句块中只能访问定义在静态语句块之前的变量，定义在它之后的变量，在前面的静态语句块可以复制，但是不能访问，同时还要保证父类的执行先于子类，然后保证多线程下的并发问题</p>
|
|
<p>最终，方法区会存储当前类类信息，包括类的静态变量、类初始化代码（定义静态变量时的赋值语句 和 静态初始化代码块）、实例变量定义、实例初始化代码（定义实例变量时的赋值语句实例代码块和构造方法）和实例方法，还有父类的类信息引用。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>类加载</category>
|
|
</categories>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 Java 的 equals 和 hashCode 方法</title>
|
|
<url>/2021/01/03/%E8%81%8A%E8%81%8A-Java-%E7%9A%84-equals-%E5%92%8C-hashCode-%E6%96%B9%E6%B3%95/</url>
|
|
<content><![CDATA[<p>Java 中的这个话题也是比较常遇到的，关于这块原先也是比较忽略的，但是仔细想想又有点遗忘了就在这里记一下<br>简单看下代码<br><code>java.lang.Object#equals</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">equals</span><span class="params">(Object obj)</span> {</span><br><span class="line"> <span class="keyword">return</span> (<span class="built_in">this</span> == obj);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>对于所有对象的父类，<code>equals</code> 方法其实对比的就是对象的地址，也就是是否是同一个对象，试想如果像 Integer 或者 String 这种，我们没有重写 <code>equals</code>，那其实就等于是在用<code>==</code>，可能就没法达到我们的目的，所以像 String 这种常用的 jdk 类都是默认重写了<br><code>java.lang.String#equals</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">equals</span><span class="params">(Object anObject)</span> {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span> == anObject) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (anObject <span class="keyword">instanceof</span> String) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">anotherString</span> <span class="operator">=</span> (String)anObject;</span><br><span class="line"> <span class="type">int</span> <span class="variable">n</span> <span class="operator">=</span> value.length;</span><br><span class="line"> <span class="keyword">if</span> (n == anotherString.value.length) {</span><br><span class="line"> <span class="type">char</span> v1[] = value;</span><br><span class="line"> <span class="type">char</span> v2[] = anotherString.value;</span><br><span class="line"> <span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">while</span> (n-- != <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">if</span> (v1[i] != v2[i])</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> i++;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后呢就是为啥一些书或者 <code>effective java</code> 中写了 <code>equals</code> 跟 <code>hashCode</code> 要一起重写，这里涉及到当对象作为 <code>HashMap</code> 的 <code>key</code> 的时候<br>首先 <code>HashMap</code> 会使用 <code>hashCode</code> 去判断是否在同一个槽里，然后在通过 <code>equals</code> 去判断是否是同一个 <code>key</code>，是的话就替换，不是的话就链表接下去，如果不重写 <code>hashCode</code> 的话，默认的 <code>object</code> 的<code>hashCode</code> 是 <code>native</code> 方法，根据对象的地址生成的，这样其实对象的值相同的话，因为地址不同，<code>HashMap</code> 也会出现异常，所以需要重写，同时也需要重写 <code>equals</code> 方法，才能确认是同一个 <code>key</code>，而不是落在同一个槽的不同 <code>key</code>.</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 Java 的类加载机制二</title>
|
|
<url>/2021/06/13/%E8%81%8A%E8%81%8A-Java-%E7%9A%84%E7%B1%BB%E5%8A%A0%E8%BD%BD%E6%9C%BA%E5%88%B6%E4%BA%8C/</url>
|
|
<content><![CDATA[<h2 id="类加载器"><a href="#类加载器" class="headerlink" title="类加载器"></a>类加载器</h2><p>类加载机制中说来说去其实也逃不开类加载器这个话题，我们就来说下类加载器这个话题，Java 在 jdk1.2 以后开始有了<br>Java 虚拟机设计团队有意把加载阶段中的“通过一个类的全限定名来获取描述该类的二进制字节流”这个动作放到 Java 虚拟机外部去实现，以便让应用程序自己去决定如何去获取所需的类。实现这个动作的代码被称为“类加载器”(Class Loader).<br>其实在 Java 中类加载器有一个很常用的作用，比如一个类的唯一性，其实是由加载它的类加载器和这个类一起来确定这个类在虚拟机的唯一性，这里也参考下周志明书里的例子</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">ClassLoaderTest</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="type">ClassLoader</span> <span class="variable">myLoader</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ClassLoader</span>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> Class<?> loadClass(String name) <span class="keyword">throws</span> ClassNotFoundException {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">fileName</span> <span class="operator">=</span> name.substring(name.lastIndexOf(<span class="string">"."</span>) + <span class="number">1</span>) + <span class="string">".class"</span>;</span><br><span class="line"> <span class="type">InputStream</span> <span class="variable">is</span> <span class="operator">=</span> getClass().getResourceAsStream(fileName);</span><br><span class="line"> <span class="keyword">if</span> (is == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">super</span>.loadClass(name);</span><br><span class="line"> }</span><br><span class="line"> <span class="type">byte</span>[] b = <span class="keyword">new</span> <span class="title class_">byte</span>[is.available()];</span><br><span class="line"> is.read(b);</span><br><span class="line"> <span class="keyword">return</span> defineClass(name, b, <span class="number">0</span>, b.length);</span><br><span class="line"> } <span class="keyword">catch</span> (IOException e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">ClassNotFoundException</span>(name);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> };</span><br><span class="line"> <span class="type">Object</span> <span class="variable">object</span> <span class="operator">=</span> myLoader.loadClass(<span class="string">"com.nicksxs.demo.ClassLoaderTest"</span>).newInstance();</span><br><span class="line"> System.out.println(object.getClass());</span><br><span class="line"> System.out.println(object <span class="keyword">instanceof</span> ClassLoaderTest);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>可以看下结果<br><img data-src="https://img.nicksxs.me/uPic/ADIJqg.png"><br>这里说明了当一个是由虚拟机的应用程序类加载器所加载的和另一个由自己写的自定义类加载器加载的，虽然是同一个类，但是 instanceof 的结果就是 false 的</p>
|
|
<h2 id="双亲委派"><a href="#双亲委派" class="headerlink" title="双亲委派"></a>双亲委派</h2><p>自 JDK1.2 以来，Java 一直有些三层类加载器、双亲委派的类加载架构</p>
|
|
<h3 id="启动类加载器"><a href="#启动类加载器" class="headerlink" title="启动类加载器"></a>启动类加载器</h3><p>首先是启动类加载器，<code>Bootstrap Class Loader</code>，这个类加载器负责加载放在<JAVA_HOME>\lib目录，或者被-Xbootclasspath参数所指定的路径中存放的，而且是Java 虚拟机能够识别的（按照文件名识别，如 rt.jar、tools.jar，名字不符合的类库即使放在 lib 目录中，也不会被加载）类库加载到虚拟机的内存中，启动类加载器无法被 Java 程序直接引用，用户在编写自定义类加载器时，如果需要把家在请求为派给引导类加载器去处理，那直接使用 null 代替即可，可以看下 java.lang.ClassLoader.getClassLoader()方法的代码片段</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Returns the class loader for the class. Some implementations may use</span></span><br><span class="line"><span class="comment"> * null to represent the bootstrap class loader. This method will return</span></span><br><span class="line"><span class="comment"> * null in such implementations if this class was loaded by the bootstrap</span></span><br><span class="line"><span class="comment"> * class loader.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p> If a security manager is present, and the caller's class loader is</span></span><br><span class="line"><span class="comment"> * not null and the caller's class loader is not the same as or an ancestor of</span></span><br><span class="line"><span class="comment"> * the class loader for the class whose class loader is requested, then</span></span><br><span class="line"><span class="comment"> * this method calls the security manager's {<span class="doctag">@code</span> checkPermission}</span></span><br><span class="line"><span class="comment"> * method with a {<span class="doctag">@code</span> RuntimePermission("getClassLoader")}</span></span><br><span class="line"><span class="comment"> * permission to ensure it's ok to access the class loader for the class.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p>If this object</span></span><br><span class="line"><span class="comment"> * represents a primitive type or void, null is returned.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> the class loader that loaded the class or interface</span></span><br><span class="line"><span class="comment"> * represented by this object.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@throws</span> SecurityException</span></span><br><span class="line"><span class="comment"> * if a security manager exists and its</span></span><br><span class="line"><span class="comment"> * {<span class="doctag">@code</span> checkPermission} method denies</span></span><br><span class="line"><span class="comment"> * access to the class loader for the class.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> java.lang.ClassLoader</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> SecurityManager#checkPermission</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> java.lang.RuntimePermission</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="meta">@CallerSensitive</span></span><br><span class="line"> <span class="keyword">public</span> ClassLoader <span class="title function_">getClassLoader</span><span class="params">()</span> {</span><br><span class="line"> <span class="type">ClassLoader</span> <span class="variable">cl</span> <span class="operator">=</span> getClassLoader0();</span><br><span class="line"> <span class="keyword">if</span> (cl == <span class="literal">null</span>)</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="type">SecurityManager</span> <span class="variable">sm</span> <span class="operator">=</span> System.getSecurityManager();</span><br><span class="line"> <span class="keyword">if</span> (sm != <span class="literal">null</span>) {</span><br><span class="line"> ClassLoader.checkClassLoaderPermission(cl, Reflection.getCallerClass());</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> cl;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<h3 id="扩展类加载器"><a href="#扩展类加载器" class="headerlink" title="扩展类加载器"></a>扩展类加载器</h3><p>这个类加载器是在类sun.misc.Launcher.ExtClassLoader中以 Java 代码的形式实现的，它负责在家<JAVA_HOME>\lib\ext 目录中，或者被 java.ext.dirs系统变量中所指定的路径中的所有类库，它其实目的是为了实现 Java 系统类库的扩展机制</p>
|
|
<h3 id="应用程序类加载器"><a href="#应用程序类加载器" class="headerlink" title="应用程序类加载器"></a>应用程序类加载器</h3><p>这个类加载器是由sun.misc.Launcher.AppClassLoader实现，通过 java 代码，并且是 ClassLoader 类中的 getSystemClassLoader()方法的返回值，可以看一下代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Returns the system class loader for delegation. This is the default</span></span><br><span class="line"><span class="comment"> * delegation parent for new <tt>ClassLoader</tt> instances, and is</span></span><br><span class="line"><span class="comment"> * typically the class loader used to start the application.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p> This method is first invoked early in the runtime's startup</span></span><br><span class="line"><span class="comment"> * sequence, at which point it creates the system class loader and sets it</span></span><br><span class="line"><span class="comment"> * as the context class loader of the invoking <tt>Thread</tt>.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p> The default system class loader is an implementation-dependent</span></span><br><span class="line"><span class="comment"> * instance of this class.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p> If the system property "<tt>java.system.class.loader</tt>" is defined</span></span><br><span class="line"><span class="comment"> * when this method is first invoked then the value of that property is</span></span><br><span class="line"><span class="comment"> * taken to be the name of a class that will be returned as the system</span></span><br><span class="line"><span class="comment"> * class loader. The class is loaded using the default system class loader</span></span><br><span class="line"><span class="comment"> * and must define a public constructor that takes a single parameter of</span></span><br><span class="line"><span class="comment"> * type <tt>ClassLoader</tt> which is used as the delegation parent. An</span></span><br><span class="line"><span class="comment"> * instance is then created using this constructor with the default system</span></span><br><span class="line"><span class="comment"> * class loader as the parameter. The resulting class loader is defined</span></span><br><span class="line"><span class="comment"> * to be the system class loader.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p> If a security manager is present, and the invoker's class loader is</span></span><br><span class="line"><span class="comment"> * not <tt>null</tt> and the invoker's class loader is not the same as or</span></span><br><span class="line"><span class="comment"> * an ancestor of the system class loader, then this method invokes the</span></span><br><span class="line"><span class="comment"> * security manager's {<span class="doctag">@link</span></span></span><br><span class="line"><span class="comment"> * SecurityManager#checkPermission(java.security.Permission)</span></span><br><span class="line"><span class="comment"> * <tt>checkPermission</tt>} method with a {<span class="doctag">@link</span></span></span><br><span class="line"><span class="comment"> * RuntimePermission#RuntimePermission(String)</span></span><br><span class="line"><span class="comment"> * <tt>RuntimePermission("getClassLoader")</tt>} permission to verify</span></span><br><span class="line"><span class="comment"> * access to the system class loader. If not, a</span></span><br><span class="line"><span class="comment"> * <tt>SecurityException</tt> will be thrown. </p></span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> The system <tt>ClassLoader</tt> for delegation, or</span></span><br><span class="line"><span class="comment"> * <tt>null</tt> if none</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@throws</span> SecurityException</span></span><br><span class="line"><span class="comment"> * If a security manager exists and its <tt>checkPermission</tt></span></span><br><span class="line"><span class="comment"> * method doesn't allow access to the system class loader.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@throws</span> IllegalStateException</span></span><br><span class="line"><span class="comment"> * If invoked recursively during the construction of the class</span></span><br><span class="line"><span class="comment"> * loader specified by the "<tt>java.system.class.loader</tt>"</span></span><br><span class="line"><span class="comment"> * property.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@throws</span> Error</span></span><br><span class="line"><span class="comment"> * If the system property "<tt>java.system.class.loader</tt>"</span></span><br><span class="line"><span class="comment"> * is defined but the named class could not be loaded, the</span></span><br><span class="line"><span class="comment"> * provider class does not define the required constructor, or an</span></span><br><span class="line"><span class="comment"> * exception is thrown by that constructor when it is invoked. The</span></span><br><span class="line"><span class="comment"> * underlying cause of the error can be retrieved via the</span></span><br><span class="line"><span class="comment"> * {<span class="doctag">@link</span> Throwable#getCause()} method.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@revised</span> 1.4</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="meta">@CallerSensitive</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> ClassLoader <span class="title function_">getSystemClassLoader</span><span class="params">()</span> {</span><br><span class="line"> initSystemClassLoader();</span><br><span class="line"> <span class="keyword">if</span> (scl == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">SecurityManager</span> <span class="variable">sm</span> <span class="operator">=</span> System.getSecurityManager();</span><br><span class="line"> <span class="keyword">if</span> (sm != <span class="literal">null</span>) {</span><br><span class="line"> checkClassLoaderPermission(scl, Reflection.getCallerClass());</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> scl;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title function_">initSystemClassLoader</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (!sclSet) {</span><br><span class="line"> <span class="keyword">if</span> (scl != <span class="literal">null</span>)</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalStateException</span>(<span class="string">"recursive invocation"</span>);</span><br><span class="line"> <span class="comment">// 主要的第一步是这</span></span><br><span class="line"> sun.misc.<span class="type">Launcher</span> <span class="variable">l</span> <span class="operator">=</span> sun.misc.Launcher.getLauncher();</span><br><span class="line"> <span class="keyword">if</span> (l != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">Throwable</span> <span class="variable">oops</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="comment">// 然后是这</span></span><br><span class="line"> scl = l.getClassLoader();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> scl = AccessController.doPrivileged(</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">SystemClassLoaderAction</span>(scl));</span><br><span class="line"> } <span class="keyword">catch</span> (PrivilegedActionException pae) {</span><br><span class="line"> oops = pae.getCause();</span><br><span class="line"> <span class="keyword">if</span> (oops <span class="keyword">instanceof</span> InvocationTargetException) {</span><br><span class="line"> oops = oops.getCause();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (oops != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">if</span> (oops <span class="keyword">instanceof</span> Error) {</span><br><span class="line"> <span class="keyword">throw</span> (Error) oops;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// wrap the exception</span></span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">Error</span>(oops);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> sclSet = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"><span class="comment">// 接着跟到sun.misc.Launcher#getClassLoader</span></span><br><span class="line"><span class="keyword">public</span> ClassLoader <span class="title function_">getClassLoader</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>.loader;</span><br><span class="line"> }</span><br><span class="line"><span class="comment">// 然后看到这 sun.misc.Launcher#Launcher</span></span><br><span class="line"><span class="keyword">public</span> <span class="title function_">Launcher</span><span class="params">()</span> {</span><br><span class="line"> Launcher.ExtClassLoader var1;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> var1 = Launcher.ExtClassLoader.getExtClassLoader();</span><br><span class="line"> } <span class="keyword">catch</span> (IOException var10) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">InternalError</span>(<span class="string">"Could not create extension class loader"</span>, var10);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 可以看到 就是 AppClassLoader</span></span><br><span class="line"> <span class="built_in">this</span>.loader = Launcher.AppClassLoader.getAppClassLoader(var1);</span><br><span class="line"> } <span class="keyword">catch</span> (IOException var9) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">InternalError</span>(<span class="string">"Could not create application class loader"</span>, var9);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> Thread.currentThread().setContextClassLoader(<span class="built_in">this</span>.loader);</span><br><span class="line"> <span class="type">String</span> <span class="variable">var2</span> <span class="operator">=</span> System.getProperty(<span class="string">"java.security.manager"</span>);</span><br><span class="line"> <span class="keyword">if</span> (var2 != <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">SecurityManager</span> <span class="variable">var3</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"> <span class="keyword">if</span> (!<span class="string">""</span>.equals(var2) && !<span class="string">"default"</span>.equals(var2)) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> var3 = (SecurityManager)<span class="built_in">this</span>.loader.loadClass(var2).newInstance();</span><br><span class="line"> } <span class="keyword">catch</span> (IllegalAccessException var5) {</span><br><span class="line"> } <span class="keyword">catch</span> (InstantiationException var6) {</span><br><span class="line"> } <span class="keyword">catch</span> (ClassNotFoundException var7) {</span><br><span class="line"> } <span class="keyword">catch</span> (ClassCastException var8) {</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> var3 = <span class="keyword">new</span> <span class="title class_">SecurityManager</span>();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (var3 == <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">InternalError</span>(<span class="string">"Could not create SecurityManager: "</span> + var2);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> System.setSecurityManager(var3);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>它负责加载用户类路径(ClassPath)上所有的类库，我们可以直接在代码中使用这个类加载器，如果我们的代码中没有自定义的类在加载器，一般情况下这个就是程序中默认的类加载器</p>
|
|
<h3 id="双亲委派模型"><a href="#双亲委派模型" class="headerlink" title="双亲委派模型"></a>双亲委派模型</h3><p><img data-src="https://img.nicksxs.me/uPic/ztS0pn.png"><br>双亲委派模型的工作过程是：如果一个类加载器收到了类加载的请求，它首先不会自己去尝试家在这个类，而是把这个请求为派给父类加载器去完成，每一个层次的类加载器都是如此，因此所有的家在请求最终都应该传送到最顶层的启动类加载器中，只有当父类加载器反馈自己无法完成加载请求（它的搜索范围中没有找到所需要的类）时，子加载器才会尝试自己去完成加载。<br>使用双亲委派模型来组织类加载器之间的关系，一个显而易见的好处就是 Java 中的类随着它的类加载器一起举杯了一种带有优先级的层次关系。例如类 java.lang.Object，它存放在 rt.jar 之中，无论哪一个类加载器要家在这个类，最终都是委派给处于模型最顶层的启动类加载器进行加载，因此 Object 类在程序的各种类加载器环境中都能够保证是同一个类。反之，如果没有使用双薪委派模型，都由各个类加载器自行去加载的话，如果用户自己也编写了一个名为 java.lang.Object 的类，并放在程序的 ClassPath 中，那系统中就会出现多个不同的 Object 类，Java 类型体系中最基础的行为也就无从保证，应用程序将会变得一片混乱。<br>可以来看下双亲委派模型的代码实现</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Loads the class with the specified <a href="#name">binary name</a>. The</span></span><br><span class="line"><span class="comment"> * default implementation of this method searches for classes in the</span></span><br><span class="line"><span class="comment"> * following order:</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <ol></span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <li><p> Invoke {<span class="doctag">@link</span> #findLoadedClass(String)} to check if the class</span></span><br><span class="line"><span class="comment"> * has already been loaded. </p></li></span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <li><p> Invoke the {<span class="doctag">@link</span> #loadClass(String) <tt>loadClass</tt>} method</span></span><br><span class="line"><span class="comment"> * on the parent class loader. If the parent is <tt>null</tt> the class</span></span><br><span class="line"><span class="comment"> * loader built-in to the virtual machine is used, instead. </p></li></span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <li><p> Invoke the {<span class="doctag">@link</span> #findClass(String)} method to find the</span></span><br><span class="line"><span class="comment"> * class. </p></li></span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * </ol></span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p> If the class was found using the above steps, and the</span></span><br><span class="line"><span class="comment"> * <tt>resolve</tt> flag is true, this method will then invoke the {<span class="doctag">@link</span></span></span><br><span class="line"><span class="comment"> * #resolveClass(Class)} method on the resulting <tt>Class</tt> object.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p> Subclasses of <tt>ClassLoader</tt> are encouraged to override {<span class="doctag">@link</span></span></span><br><span class="line"><span class="comment"> * #findClass(String)}, rather than this method. </p></span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <p> Unless overridden, this method synchronizes on the result of</span></span><br><span class="line"><span class="comment"> * {<span class="doctag">@link</span> #getClassLoadingLock <tt>getClassLoadingLock</tt>} method</span></span><br><span class="line"><span class="comment"> * during the entire class loading process.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> name</span></span><br><span class="line"><span class="comment"> * The <a href="#name">binary name</a> of the class</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> resolve</span></span><br><span class="line"><span class="comment"> * If <tt>true</tt> then resolve the class</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> The resulting <tt>Class</tt> object</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@throws</span> ClassNotFoundException</span></span><br><span class="line"><span class="comment"> * If the class could not be found</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">protected</span> Class<?> loadClass(String name, <span class="type">boolean</span> resolve)</span><br><span class="line"> <span class="keyword">throws</span> ClassNotFoundException</span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">synchronized</span> (getClassLoadingLock(name)) {</span><br><span class="line"> <span class="comment">// First, check if the class has already been loaded</span></span><br><span class="line"> Class<?> c = findLoadedClass(name);</span><br><span class="line"> <span class="keyword">if</span> (c == <span class="literal">null</span>) {</span><br><span class="line"> <span class="type">long</span> <span class="variable">t0</span> <span class="operator">=</span> System.nanoTime();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">if</span> (parent != <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// 委托父类加载</span></span><br><span class="line"> c = parent.loadClass(name, <span class="literal">false</span>);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="comment">// 使用启动类加载器</span></span><br><span class="line"> c = findBootstrapClassOrNull(name);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (ClassNotFoundException e) {</span><br><span class="line"> <span class="comment">// ClassNotFoundException thrown if class not found</span></span><br><span class="line"> <span class="comment">// from the non-null parent class loader</span></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (c == <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// If still not found, then invoke findClass in order</span></span><br><span class="line"> <span class="comment">// to find the class.</span></span><br><span class="line"> <span class="type">long</span> <span class="variable">t1</span> <span class="operator">=</span> System.nanoTime();</span><br><span class="line"> <span class="comment">// 调用自己的 findClass() 方法尝试进行加载</span></span><br><span class="line"> c = findClass(name);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// this is the defining class loader; record the stats</span></span><br><span class="line"> sun.misc.PerfCounter.getParentDelegationTime().addTime(t1 - t0);</span><br><span class="line"> sun.misc.PerfCounter.getFindClassTime().addElapsedTimeFrom(t1);</span><br><span class="line"> sun.misc.PerfCounter.getFindClasses().increment();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (resolve) {</span><br><span class="line"> resolveClass(c);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> c;</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<h3 id="破坏双亲委派"><a href="#破坏双亲委派" class="headerlink" title="破坏双亲委派"></a>破坏双亲委派</h3><p>关于破坏双亲委派模型，第一次是在 JDK1.2 之后引入了双亲委派模型之前，那么在那之前已经有了类加载器，所以java.lang.ClassLoader 中添加了一个 protected 方法 findClass()，并引导用户编写的类加载逻辑时尽可能去重写这个方法，而不是在 loadClass()中编写代码。这个跟上面的逻辑其实类似，当父类加载失败，会调用 findClass()来完成加载；第二次是因为这个模型本身还有一些不足之处，比如 SPI 这种，所以有设计了线程下上下文类加载器(Thread Context ClassLoader)。这个类加载器可以通过 java.lang.Thread 类的 java.lang.Thread#setContextClassLoader() 进行设置，然后第三种是为了追求程序动态性，这里有涉及到了 osgi 等概念，就不展开了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>类加载</tag>
|
|
<tag>加载</tag>
|
|
<tag>验证</tag>
|
|
<tag>准备</tag>
|
|
<tag>解析</tag>
|
|
<tag>初始化</tag>
|
|
<tag>链接</tag>
|
|
<tag>双亲委派</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 Linux 下的 top 命令</title>
|
|
<url>/2021/03/28/%E8%81%8A%E8%81%8A-Linux-%E4%B8%8B%E7%9A%84-top-%E5%91%BD%E4%BB%A4/</url>
|
|
<content><![CDATA[<p>top 命令在日常的 Linux 使用中，特别是做一些服务器的简单状态查看，排查故障都起了比较大的作用，但是由于这个命令看到的东西比较多，一般只会看部分，或者说像我这样就会比较片面地看一些信息，比如默认是进程维度的，可以在启动命令的时候加<code>-H</code>进入线程模式</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">-H :Threads-mode operation</span><br><span class="line"> Instructs top to display individual threads. Without this command-line option a summation of all threads in each process is shown. Later</span><br><span class="line"> this can be changed with the `H' interactive command.</span><br></pre></td></tr></table></figure>
|
|
<p>这样就能用在 Java 中去 jstack 中找到对应的线程<br>其实还有比较重要的两个操作，<br>一个是在 top 启动状态下，按<code>c</code>键，这样能把比如说是一个 Java 进程，具体的进程命令显示出来<br>像这样<br>执行前是这样<br><img data-src="https://img.nicksxs.me/uPic/LKn8Bs.png"><br>执行后是这样<br><img data-src="https://img.nicksxs.me/uPic/1xD6VM.png"><br>第二个就是排序了</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">SORTING of task window</span><br><span class="line"></span><br><span class="line"> For compatibility, this top supports most of the former top sort keys. Since this is primarily a service to former top users, these commands</span><br><span class="line"> do not appear on any help screen.</span><br><span class="line"> command sorted-field supported</span><br><span class="line"> A start time (non-display) No</span><br><span class="line"> M %MEM Yes</span><br><span class="line"> N PID Yes</span><br><span class="line"> P %CPU Yes</span><br><span class="line"> T TIME+ Yes</span><br><span class="line"></span><br><span class="line"> Before using any of the following sort provisions, top suggests that you temporarily turn on column highlighting using the `x' interactive com‐</span><br><span class="line"> mand. That will help ensure that the actual sort environment matches your intent.</span><br><span class="line"></span><br><span class="line"> The following interactive commands will only be honored when the current sort field is visible. The sort field might not be visible because:</span><br><span class="line"> 1) there is insufficient Screen Width</span><br><span class="line"> 2) the `f' interactive command turned it Off</span><br><span class="line"></span><br><span class="line"> < :Move-Sort-Field-Left</span><br><span class="line"> Moves the sort column to the left unless the current sort field is the first field being displayed.</span><br><span class="line"></span><br><span class="line"> > :Move-Sort-Field-Right</span><br><span class="line"> Moves the sort column to the right unless the current sort field is the last field being displayed.</span><br></pre></td></tr></table></figure>
|
|
<p>查看 man page 可以找到这一段，其实一般 man page 都是最细致的，只不过因为太多了，有时候懒得看，这里可以通过大写 <code>M</code> 和大写 <code>P</code> 分别按内存和 CPU 排序，下面还有两个小技巧，通过按 x 可以将当前活跃的排序列用不同颜色标出来，然后可以通过<code><</code>和<code>></code>直接左右移动排序列</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Linux</category>
|
|
<category>命令</category>
|
|
<category>小技巧</category>
|
|
<category>top</category>
|
|
<category>top</category>
|
|
<category>排序</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>排序</tag>
|
|
<tag>linux</tag>
|
|
<tag>小技巧</tag>
|
|
<tag>top</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 Java 自带的那些*逆天*工具</title>
|
|
<url>/2020/08/02/%E8%81%8A%E8%81%8A-Java-%E8%87%AA%E5%B8%A6%E7%9A%84%E9%82%A3%E4%BA%9B%E9%80%86%E5%A4%A9%E5%B7%A5%E5%85%B7/</url>
|
|
<content><![CDATA[<p>原谅我的标题党，其实这些工具的确很厉害，之前其实介绍过一点相关的，是从我一次问题排查的过程中用到的，但是最近又有碰到一次排查问题，发现其实用 idea 直接 <code>dump thread</code> 是不现实的，毕竟服务器环境的没法这么操作，那就得用 Java 的那些工具了</p>
|
|
<h3 id="jstack-jps"><a href="#jstack-jps" class="headerlink" title="jstack & jps"></a>jstack & jps</h3><p>譬如 <code>jstack</code>，这个命令其实不能更简单了<br>看看 help 信息<br><img data-src="https://img.nicksxs.me/uPic/1eYvcu.png"><br>用<code>-l</code>参数可以打出锁的额外信息，然后后面的 pid 就是进程 id 咯，机智的小伙伴会问了（就你这个小白才问这么蠢的问题🤦♂️），怎么看 Java 应用的进程呢<br>那就是 <code>jps</code> 了，命令也很简单，一般直接用 <code>jps</code>命令就好了，不过也可以 help 看一下<br><img data-src="https://img.nicksxs.me/uPic/DqWZe1.png"><br>稍微解释下，-q是只显示进程 id，-m是输出给main 方法的参数，比如我在配置中加给参数<br><img data-src="https://img.nicksxs.me/uPic/4A0pqD.png"><br>然后用 <code>jps -m</code>查看<br><img data-src="https://img.nicksxs.me/uPic/LlGPfU.png"><br><code>-v</code>加上小 v 的话就是打印 jvm 参数<br><img data-src="https://img.nicksxs.me/uPic/3UTgsd.png"><br>还是有点东西，然后就继续介绍 jstack 了，然后我们看看 jstack 出来是啥，为了加点内容我加了个死锁</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> <span class="keyword">throws</span> InterruptedException {</span><br><span class="line"> SpringApplication.run(ThreadDumpDemoApplication.class, args);</span><br><span class="line"> <span class="type">ReentrantLock</span> <span class="variable">lock1</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ReentrantLock</span>();</span><br><span class="line"> <span class="type">ReentrantLock</span> <span class="variable">lock2</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ReentrantLock</span>();</span><br><span class="line"> <span class="type">Thread</span> <span class="variable">t1</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Thread</span>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> lock1.lock();</span><br><span class="line"> TimeUnit.SECONDS.sleep(<span class="number">1</span>);</span><br><span class="line"> lock2.lock();</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> };</span><br><span class="line"> <span class="type">Thread</span> <span class="variable">t2</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Thread</span>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> lock2.lock();</span><br><span class="line"> TimeUnit.SECONDS.sleep(<span class="number">1</span>);</span><br><span class="line"> lock1.lock();</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> };</span><br><span class="line"> t1.setName(<span class="string">"mythread1"</span>);</span><br><span class="line"> t2.setName(<span class="string">"mythread2"</span>);</span><br><span class="line"> t1.start();</span><br><span class="line"> t2.start();</span><br><span class="line"> Thread.sleep(<span class="number">10000</span>);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后看看出来时怎么样的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="number">2020</span>-08-<span class="number">02</span> <span class="number">21</span>:<span class="number">50</span>:<span class="number">32</span></span><br><span class="line">Full thread dump Java <span class="title function_">HotSpot</span><span class="params">(TM)</span> <span class="number">64</span>-Bit Server <span class="title function_">VM</span> <span class="params">(<span class="number">25.201</span>-b09 mixed mode)</span>:</span><br><span class="line"></span><br><span class="line"><span class="string">"DestroyJavaVM"</span> #<span class="number">147</span> prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9dd807000</span> nid=<span class="number">0x2603</span> waiting on condition [<span class="number">0x0000000000000000</span>]</span><br><span class="line"> java.lang.Thread.State: RUNNABLE</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"mythread2"</span> #<span class="number">140</span> prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9dd877000</span> nid=<span class="number">0x9903</span> waiting on condition [<span class="number">0x0000700006fb9000</span>]</span><br><span class="line"> java.lang.Thread.State: WAITING (parking)</span><br><span class="line"> at sun.misc.Unsafe.park(Native Method)</span><br><span class="line"> - parking to wait <span class="keyword">for</span> <<span class="number">0x000000076f5d4330</span>> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)</span><br><span class="line"> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:<span class="number">175</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:<span class="number">836</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:<span class="number">870</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:<span class="number">1199</span>)</span><br><span class="line"> at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:<span class="number">209</span>)</span><br><span class="line"> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:<span class="number">285</span>)</span><br><span class="line"> at com.nicksxs.thread_dump_demo.ThreadDumpDemoApplication$<span class="number">2.</span>run(ThreadDumpDemoApplication.java:<span class="number">34</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - <<span class="number">0x000000076f5d4360</span>> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)</span><br><span class="line"></span><br><span class="line"><span class="string">"mythread1"</span> #<span class="number">139</span> prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9de873800</span> nid=<span class="number">0x9a03</span> waiting on condition [<span class="number">0x0000700006eb6000</span>]</span><br><span class="line"> java.lang.Thread.State: WAITING (parking)</span><br><span class="line"> at sun.misc.Unsafe.park(Native Method)</span><br><span class="line"> - parking to wait <span class="keyword">for</span> <<span class="number">0x000000076f5d4360</span>> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)</span><br><span class="line"> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:<span class="number">175</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:<span class="number">836</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:<span class="number">870</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:<span class="number">1199</span>)</span><br><span class="line"> at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:<span class="number">209</span>)</span><br><span class="line"> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:<span class="number">285</span>)</span><br><span class="line"> at com.nicksxs.thread_dump_demo.ThreadDumpDemoApplication$<span class="number">1.</span>run(ThreadDumpDemoApplication.java:<span class="number">22</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - <<span class="number">0x000000076f5d4330</span>> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)</span><br><span class="line"></span><br><span class="line"><span class="string">"http-nio-8080-Acceptor"</span> #<span class="number">137</span> daemon prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9de1ac000</span> nid=<span class="number">0x9b03</span> runnable [<span class="number">0x0000700006db3000</span>]</span><br><span class="line"> java.lang.Thread.State: RUNNABLE</span><br><span class="line"> at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)</span><br><span class="line"> at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:<span class="number">422</span>)</span><br><span class="line"> at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:<span class="number">250</span>)</span><br><span class="line"> - locked <<span class="number">0x000000076f1e4820</span>> (a java.lang.Object)</span><br><span class="line"> at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:<span class="number">469</span>)</span><br><span class="line"> at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:<span class="number">71</span>)</span><br><span class="line"> at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:<span class="number">95</span>)</span><br><span class="line"> at java.lang.Thread.run(Thread.java:<span class="number">748</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"http-nio-8080-ClientPoller"</span> #<span class="number">136</span> daemon prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9dd876800</span> nid=<span class="number">0x6503</span> runnable [<span class="number">0x0000700006cb0000</span>]</span><br><span class="line"> java.lang.Thread.State: RUNNABLE</span><br><span class="line"> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)</span><br><span class="line"> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:<span class="number">198</span>)</span><br><span class="line"> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:<span class="number">117</span>)</span><br><span class="line"> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:<span class="number">86</span>)</span><br><span class="line"> - locked <<span class="number">0x000000076f2978c8</span>> (a sun.nio.ch.Util$<span class="number">3</span>)</span><br><span class="line"> - locked <<span class="number">0x000000076f2978b8</span>> (a java.util.Collections$UnmodifiableSet)</span><br><span class="line"> - locked <<span class="number">0x000000076f297798</span>> (a sun.nio.ch.KQueueSelectorImpl)</span><br><span class="line"> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:<span class="number">97</span>)</span><br><span class="line"> at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:<span class="number">709</span>)</span><br><span class="line"> at java.lang.Thread.run(Thread.java:<span class="number">748</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"http-nio-8080-exec-10"</span> #<span class="number">135</span> daemon prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9de1af000</span> nid=<span class="number">0x9d03</span> waiting on condition [<span class="number">0x0000700006bad000</span>]</span><br><span class="line"> java.lang.Thread.State: WAITING (parking)</span><br><span class="line"> at sun.misc.Unsafe.park(Native Method)</span><br><span class="line"> - parking to wait <span class="keyword">for</span> <<span class="number">0x000000076f26aa00</span>> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)</span><br><span class="line"> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:<span class="number">175</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:<span class="number">2039</span>)</span><br><span class="line"> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:<span class="number">442</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">107</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">33</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:<span class="number">1074</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:<span class="number">1134</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:<span class="number">624</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:<span class="number">61</span>)</span><br><span class="line"> at java.lang.Thread.run(Thread.java:<span class="number">748</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"http-nio-8080-exec-9"</span> #<span class="number">134</span> daemon prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9de1ab800</span> nid=<span class="number">0x6403</span> waiting on condition [<span class="number">0x0000700006aaa000</span>]</span><br><span class="line"> java.lang.Thread.State: WAITING (parking)</span><br><span class="line"> at sun.misc.Unsafe.park(Native Method)</span><br><span class="line"> - parking to wait <span class="keyword">for</span> <<span class="number">0x000000076f26aa00</span>> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)</span><br><span class="line"> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:<span class="number">175</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:<span class="number">2039</span>)</span><br><span class="line"> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:<span class="number">442</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">107</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">33</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:<span class="number">1074</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:<span class="number">1134</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:<span class="number">624</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:<span class="number">61</span>)</span><br><span class="line"> at java.lang.Thread.run(Thread.java:<span class="number">748</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"http-nio-8080-exec-8"</span> #<span class="number">133</span> daemon prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9de873000</span> nid=<span class="number">0x9f03</span> waiting on condition [<span class="number">0x00007000069a7000</span>]</span><br><span class="line"> java.lang.Thread.State: WAITING (parking)</span><br><span class="line"> at sun.misc.Unsafe.park(Native Method)</span><br><span class="line"> - parking to wait <span class="keyword">for</span> <<span class="number">0x000000076f26aa00</span>> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)</span><br><span class="line"> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:<span class="number">175</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:<span class="number">2039</span>)</span><br><span class="line"> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:<span class="number">442</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">107</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">33</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:<span class="number">1074</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:<span class="number">1134</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:<span class="number">624</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:<span class="number">61</span>)</span><br><span class="line"> at java.lang.Thread.run(Thread.java:<span class="number">748</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"http-nio-8080-exec-7"</span> #<span class="number">132</span> daemon prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9df0a1800</span> nid=<span class="number">0xa103</span> waiting on condition [<span class="number">0x00007000068a4000</span>]</span><br><span class="line"> java.lang.Thread.State: WAITING (parking)</span><br><span class="line"> at sun.misc.Unsafe.park(Native Method)</span><br><span class="line"> - parking to wait <span class="keyword">for</span> <<span class="number">0x000000076f26aa00</span>> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)</span><br><span class="line"> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:<span class="number">175</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:<span class="number">2039</span>)</span><br><span class="line"> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:<span class="number">442</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">107</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">33</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:<span class="number">1074</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:<span class="number">1134</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:<span class="number">624</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:<span class="number">61</span>)</span><br><span class="line"> at java.lang.Thread.run(Thread.java:<span class="number">748</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"http-nio-8080-exec-6"</span> #<span class="number">131</span> daemon prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9df242800</span> nid=<span class="number">0x6103</span> waiting on condition [<span class="number">0x00007000067a1000</span>]</span><br><span class="line"> java.lang.Thread.State: WAITING (parking)</span><br><span class="line"> at sun.misc.Unsafe.park(Native Method)</span><br><span class="line"> - parking to wait <span class="keyword">for</span> <<span class="number">0x000000076f26aa00</span>> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)</span><br><span class="line"> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:<span class="number">175</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:<span class="number">2039</span>)</span><br><span class="line"> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:<span class="number">442</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">107</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">33</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:<span class="number">1074</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:<span class="number">1134</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:<span class="number">624</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:<span class="number">61</span>)</span><br><span class="line"> at java.lang.Thread.run(Thread.java:<span class="number">748</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"http-nio-8080-exec-5"</span> #<span class="number">130</span> daemon prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9de872000</span> nid=<span class="number">0x5f03</span> waiting on condition [<span class="number">0x000070000669e000</span>]</span><br><span class="line"> java.lang.Thread.State: WAITING (parking)</span><br><span class="line"> at sun.misc.Unsafe.park(Native Method)</span><br><span class="line"> - parking to wait <span class="keyword">for</span> <<span class="number">0x000000076f26aa00</span>> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)</span><br><span class="line"> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:<span class="number">175</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:<span class="number">2039</span>)</span><br><span class="line"> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:<span class="number">442</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">107</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">33</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:<span class="number">1074</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:<span class="number">1134</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:<span class="number">624</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:<span class="number">61</span>)</span><br><span class="line"> at java.lang.Thread.run(Thread.java:<span class="number">748</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"http-nio-8080-exec-4"</span> #<span class="number">129</span> daemon prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9de1a6000</span> nid=<span class="number">0x5e03</span> waiting on condition [<span class="number">0x000070000659b000</span>]</span><br><span class="line"> java.lang.Thread.State: WAITING (parking)</span><br><span class="line"> at sun.misc.Unsafe.park(Native Method)</span><br><span class="line"> - parking to wait <span class="keyword">for</span> <<span class="number">0x000000076f26aa00</span>> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)</span><br><span class="line"> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:<span class="number">175</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:<span class="number">2039</span>)</span><br><span class="line"> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:<span class="number">442</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">107</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">33</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:<span class="number">1074</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:<span class="number">1134</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:<span class="number">624</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:<span class="number">61</span>)</span><br><span class="line"> at java.lang.Thread.run(Thread.java:<span class="number">748</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"http-nio-8080-exec-3"</span> #<span class="number">128</span> daemon prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9de871800</span> nid=<span class="number">0x5c03</span> waiting on condition [<span class="number">0x0000700006498000</span>]</span><br><span class="line"> java.lang.Thread.State: WAITING (parking)</span><br><span class="line"> at sun.misc.Unsafe.park(Native Method)</span><br><span class="line"> - parking to wait <span class="keyword">for</span> <<span class="number">0x000000076f26aa00</span>> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)</span><br><span class="line"> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:<span class="number">175</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:<span class="number">2039</span>)</span><br><span class="line"> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:<span class="number">442</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">107</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">33</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:<span class="number">1074</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:<span class="number">1134</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:<span class="number">624</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:<span class="number">61</span>)</span><br><span class="line"> at java.lang.Thread.run(Thread.java:<span class="number">748</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"http-nio-8080-exec-2"</span> #<span class="number">127</span> daemon prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9dead9000</span> nid=<span class="number">0x5b03</span> waiting on condition [<span class="number">0x0000700006395000</span>]</span><br><span class="line"> java.lang.Thread.State: WAITING (parking)</span><br><span class="line"> at sun.misc.Unsafe.park(Native Method)</span><br><span class="line"> - parking to wait <span class="keyword">for</span> <<span class="number">0x000000076f26aa00</span>> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)</span><br><span class="line"> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:<span class="number">175</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:<span class="number">2039</span>)</span><br><span class="line"> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:<span class="number">442</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">107</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">33</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:<span class="number">1074</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:<span class="number">1134</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:<span class="number">624</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:<span class="number">61</span>)</span><br><span class="line"> at java.lang.Thread.run(Thread.java:<span class="number">748</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"http-nio-8080-exec-1"</span> #<span class="number">126</span> daemon prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9ddb00000</span> nid=<span class="number">0x5a03</span> waiting on condition [<span class="number">0x0000700006292000</span>]</span><br><span class="line"> java.lang.Thread.State: WAITING (parking)</span><br><span class="line"> at sun.misc.Unsafe.park(Native Method)</span><br><span class="line"> - parking to wait <span class="keyword">for</span> <<span class="number">0x000000076f26aa00</span>> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)</span><br><span class="line"> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:<span class="number">175</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:<span class="number">2039</span>)</span><br><span class="line"> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:<span class="number">442</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">107</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:<span class="number">33</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:<span class="number">1074</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:<span class="number">1134</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:<span class="number">624</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:<span class="number">61</span>)</span><br><span class="line"> at java.lang.Thread.run(Thread.java:<span class="number">748</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"http-nio-8080-BlockPoller"</span> #<span class="number">125</span> daemon prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9df242000</span> nid=<span class="number">0xa503</span> runnable [<span class="number">0x000070000618f000</span>]</span><br><span class="line"> java.lang.Thread.State: RUNNABLE</span><br><span class="line"> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)</span><br><span class="line"> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:<span class="number">198</span>)</span><br><span class="line"> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:<span class="number">117</span>)</span><br><span class="line"> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:<span class="number">86</span>)</span><br><span class="line"> - locked <<span class="number">0x000000076f1eea30</span>> (a sun.nio.ch.Util$<span class="number">3</span>)</span><br><span class="line"> - locked <<span class="number">0x000000076f1ee198</span>> (a java.util.Collections$UnmodifiableSet)</span><br><span class="line"> - locked <<span class="number">0x000000076f1ee010</span>> (a sun.nio.ch.KQueueSelectorImpl)</span><br><span class="line"> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:<span class="number">97</span>)</span><br><span class="line"> at org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.run(NioBlockingSelector.java:<span class="number">313</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"container-0"</span> #<span class="number">124</span> prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9df06a000</span> nid=<span class="number">0x5803</span> waiting on condition [<span class="number">0x000070000608c000</span>]</span><br><span class="line"> java.lang.Thread.State: TIMED_WAITING (sleeping)</span><br><span class="line"> at java.lang.Thread.sleep(Native Method)</span><br><span class="line"> at org.apache.catalina.core.StandardServer.await(StandardServer.java:<span class="number">570</span>)</span><br><span class="line"> at org.springframework.boot.web.embedded.tomcat.TomcatWebServer$<span class="number">1.</span>run(TomcatWebServer.java:<span class="number">197</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"Catalina-utility-2"</span> #<span class="number">123</span> prio=<span class="number">1</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9de886000</span> nid=<span class="number">0xa80f</span> waiting on condition [<span class="number">0x0000700005f89000</span>]</span><br><span class="line"> java.lang.Thread.State: WAITING (parking)</span><br><span class="line"> at sun.misc.Unsafe.park(Native Method)</span><br><span class="line"> - parking to wait <span class="keyword">for</span> <<span class="number">0x000000076c88ab58</span>> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)</span><br><span class="line"> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:<span class="number">175</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:<span class="number">2039</span>)</span><br><span class="line"> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:<span class="number">1088</span>)</span><br><span class="line"> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:<span class="number">809</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:<span class="number">1074</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:<span class="number">1134</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:<span class="number">624</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:<span class="number">61</span>)</span><br><span class="line"> at java.lang.Thread.run(Thread.java:<span class="number">748</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"Catalina-utility-1"</span> #<span class="number">122</span> prio=<span class="number">1</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9de884000</span> nid=<span class="number">0x5667</span> waiting on condition [<span class="number">0x0000700005e86000</span>]</span><br><span class="line"> java.lang.Thread.State: TIMED_WAITING (parking)</span><br><span class="line"> at sun.misc.Unsafe.park(Native Method)</span><br><span class="line"> - parking to wait <span class="keyword">for</span> <<span class="number">0x000000076c88ab58</span>> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)</span><br><span class="line"> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:<span class="number">215</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:<span class="number">2078</span>)</span><br><span class="line"> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:<span class="number">1093</span>)</span><br><span class="line"> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:<span class="number">809</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:<span class="number">1074</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:<span class="number">1134</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:<span class="number">624</span>)</span><br><span class="line"> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:<span class="number">61</span>)</span><br><span class="line"> at java.lang.Thread.run(Thread.java:<span class="number">748</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"RMI Scheduler(0)"</span> #<span class="number">15</span> daemon prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9de9ee000</span> nid=<span class="number">0x5503</span> waiting on condition [<span class="number">0x0000700005d83000</span>]</span><br><span class="line"> java.lang.Thread.State: TIMED_WAITING (parking)</span><br><span class="line"> at sun.misc.Unsafe.park(Native Method)</span><br><span class="line"> - parking to wait <span class="keyword">for</span> <<span class="number">0x00000006c0015410</span>> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)</span><br><span class="line"> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:<span class="number">215</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:<span class="number">2078</span>)</span><br><span class="line"> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:<span class="number">1093</span>)</span><br><span class="line"> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:<span class="number">809</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:<span class="number">1074</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:<span class="number">1134</span>)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:<span class="number">624</span>)</span><br><span class="line"> at java.lang.Thread.run(Thread.java:<span class="number">748</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"Attach Listener"</span> #<span class="number">13</span> daemon prio=<span class="number">9</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9df149800</span> nid=<span class="number">0x3c07</span> waiting on condition [<span class="number">0x0000000000000000</span>]</span><br><span class="line"> java.lang.Thread.State: RUNNABLE</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"RMI TCP Accept-0"</span> #<span class="number">11</span> daemon prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9df100000</span> nid=<span class="number">0x4003</span> runnable [<span class="number">0x0000700005977000</span>]</span><br><span class="line"> java.lang.Thread.State: RUNNABLE</span><br><span class="line"> at java.net.PlainSocketImpl.socketAccept(Native Method)</span><br><span class="line"> at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:<span class="number">409</span>)</span><br><span class="line"> at java.net.ServerSocket.implAccept(ServerSocket.java:<span class="number">545</span>)</span><br><span class="line"> at java.net.ServerSocket.accept(ServerSocket.java:<span class="number">513</span>)</span><br><span class="line"> at sun.management.jmxremote.LocalRMIServerSocketFactory$<span class="number">1.</span>accept(LocalRMIServerSocketFactory.java:<span class="number">52</span>)</span><br><span class="line"> at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:<span class="number">405</span>)</span><br><span class="line"> at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:<span class="number">377</span>)</span><br><span class="line"> at java.lang.Thread.run(Thread.java:<span class="number">748</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"Service Thread"</span> #<span class="number">9</span> daemon prio=<span class="number">9</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9df0ce800</span> nid=<span class="number">0x4103</span> runnable [<span class="number">0x0000000000000000</span>]</span><br><span class="line"> java.lang.Thread.State: RUNNABLE</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"C1 CompilerThread2"</span> #<span class="number">8</span> daemon prio=<span class="number">9</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9df0ce000</span> nid=<span class="number">0x4203</span> waiting on condition [<span class="number">0x0000000000000000</span>]</span><br><span class="line"> java.lang.Thread.State: RUNNABLE</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"C2 CompilerThread1"</span> #<span class="number">7</span> daemon prio=<span class="number">9</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9de0a3800</span> nid=<span class="number">0x3503</span> waiting on condition [<span class="number">0x0000000000000000</span>]</span><br><span class="line"> java.lang.Thread.State: RUNNABLE</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"C2 CompilerThread0"</span> #<span class="number">6</span> daemon prio=<span class="number">9</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9de89b000</span> nid=<span class="number">0x3403</span> waiting on condition [<span class="number">0x0000000000000000</span>]</span><br><span class="line"> java.lang.Thread.State: RUNNABLE</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"Monitor Ctrl-Break"</span> #<span class="number">5</span> daemon prio=<span class="number">5</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9df0ca000</span> nid=<span class="number">0x3303</span> runnable [<span class="number">0x0000700005468000</span>]</span><br><span class="line"> java.lang.Thread.State: RUNNABLE</span><br><span class="line"> at java.net.SocketInputStream.socketRead0(Native Method)</span><br><span class="line"> at java.net.SocketInputStream.socketRead(SocketInputStream.java:<span class="number">116</span>)</span><br><span class="line"> at java.net.SocketInputStream.read(SocketInputStream.java:<span class="number">171</span>)</span><br><span class="line"> at java.net.SocketInputStream.read(SocketInputStream.java:<span class="number">141</span>)</span><br><span class="line"> at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:<span class="number">284</span>)</span><br><span class="line"> at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:<span class="number">326</span>)</span><br><span class="line"> at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:<span class="number">178</span>)</span><br><span class="line"> - locked <<span class="number">0x00000006c001b760</span>> (a java.io.InputStreamReader)</span><br><span class="line"> at java.io.InputStreamReader.read(InputStreamReader.java:<span class="number">184</span>)</span><br><span class="line"> at java.io.BufferedReader.fill(BufferedReader.java:<span class="number">161</span>)</span><br><span class="line"> at java.io.BufferedReader.readLine(BufferedReader.java:<span class="number">324</span>)</span><br><span class="line"> - locked <<span class="number">0x00000006c001b760</span>> (a java.io.InputStreamReader)</span><br><span class="line"> at java.io.BufferedReader.readLine(BufferedReader.java:<span class="number">389</span>)</span><br><span class="line"> at com.intellij.rt.execution.application.AppMainV2$<span class="number">1.</span>run(AppMainV2.java:<span class="number">64</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"Signal Dispatcher"</span> #<span class="number">4</span> daemon prio=<span class="number">9</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9de824000</span> nid=<span class="number">0x4503</span> runnable [<span class="number">0x0000000000000000</span>]</span><br><span class="line"> java.lang.Thread.State: RUNNABLE</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"Finalizer"</span> #<span class="number">3</span> daemon prio=<span class="number">8</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9dd811800</span> nid=<span class="number">0x4f03</span> in Object.wait() [<span class="number">0x0000700005262000</span>]</span><br><span class="line"> java.lang.Thread.State: WAITING (on object monitor)</span><br><span class="line"> at java.lang.Object.wait(Native Method)</span><br><span class="line"> - waiting on <<span class="number">0x00000006c0008348</span>> (a java.lang.ref.ReferenceQueue$Lock)</span><br><span class="line"> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:<span class="number">144</span>)</span><br><span class="line"> - locked <<span class="number">0x00000006c0008348</span>> (a java.lang.ref.ReferenceQueue$Lock)</span><br><span class="line"> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:<span class="number">165</span>)</span><br><span class="line"> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:<span class="number">216</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"Reference Handler"</span> #<span class="number">2</span> daemon prio=<span class="number">10</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9de02a000</span> nid=<span class="number">0x5003</span> in Object.wait() [<span class="number">0x000070000515f000</span>]</span><br><span class="line"> java.lang.Thread.State: WAITING (on object monitor)</span><br><span class="line"> at java.lang.Object.wait(Native Method)</span><br><span class="line"> - waiting on <<span class="number">0x00000006c001b940</span>> (a java.lang.ref.Reference$Lock)</span><br><span class="line"> at java.lang.Object.wait(Object.java:<span class="number">502</span>)</span><br><span class="line"> at java.lang.ref.Reference.tryHandlePending(Reference.java:<span class="number">191</span>)</span><br><span class="line"> - locked <<span class="number">0x00000006c001b940</span>> (a java.lang.ref.Reference$Lock)</span><br><span class="line"> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:<span class="number">153</span>)</span><br><span class="line"></span><br><span class="line"> Locked ownable synchronizers:</span><br><span class="line"> - None</span><br><span class="line"></span><br><span class="line"><span class="string">"VM Thread"</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9df00b800</span> nid=<span class="number">0x2c03</span> runnable </span><br><span class="line"></span><br><span class="line"><span class="string">"GC task thread#0 (ParallelGC)"</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9de805000</span> nid=<span class="number">0x1e07</span> runnable </span><br><span class="line"></span><br><span class="line"><span class="string">"GC task thread#1 (ParallelGC)"</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9de003800</span> nid=<span class="number">0x2a03</span> runnable </span><br><span class="line"></span><br><span class="line"><span class="string">"GC task thread#2 (ParallelGC)"</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9df002000</span> nid=<span class="number">0x5403</span> runnable </span><br><span class="line"></span><br><span class="line"><span class="string">"GC task thread#3 (ParallelGC)"</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9df002800</span> nid=<span class="number">0x5203</span> runnable </span><br><span class="line"></span><br><span class="line"><span class="string">"VM Periodic Task Thread"</span> os_prio=<span class="number">31</span> tid=<span class="number">0x00007fc9df11a800</span> nid=<span class="number">0x3a03</span> waiting on condition </span><br><span class="line"></span><br><span class="line">JNI global references: <span class="number">1087</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line">Found one Java-level deadlock:</span><br><span class="line">=============================</span><br><span class="line"><span class="string">"mythread2"</span>:</span><br><span class="line"> waiting <span class="keyword">for</span> ownable synchronizer <span class="number">0x000000076f5d4330</span>, (a java.util.concurrent.locks.ReentrantLock$NonfairSync),</span><br><span class="line"> which is held by <span class="string">"mythread1"</span></span><br><span class="line"><span class="string">"mythread1"</span>:</span><br><span class="line"> waiting <span class="keyword">for</span> ownable synchronizer <span class="number">0x000000076f5d4360</span>, (a java.util.concurrent.locks.ReentrantLock$NonfairSync),</span><br><span class="line"> which is held by <span class="string">"mythread2"</span></span><br><span class="line"></span><br><span class="line">Java stack information <span class="keyword">for</span> the threads listed above:</span><br><span class="line">===================================================</span><br><span class="line"><span class="string">"mythread2"</span>:</span><br><span class="line"> at sun.misc.Unsafe.park(Native Method)</span><br><span class="line"> - parking to wait <span class="keyword">for</span> <<span class="number">0x000000076f5d4330</span>> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)</span><br><span class="line"> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:<span class="number">175</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:<span class="number">836</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:<span class="number">870</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:<span class="number">1199</span>)</span><br><span class="line"> at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:<span class="number">209</span>)</span><br><span class="line"> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:<span class="number">285</span>)</span><br><span class="line"> at com.nicksxs.thread_dump_demo.ThreadDumpDemoApplication$<span class="number">2.</span>run(ThreadDumpDemoApplication.java:<span class="number">34</span>)</span><br><span class="line"><span class="string">"mythread1"</span>:</span><br><span class="line"> at sun.misc.Unsafe.park(Native Method)</span><br><span class="line"> - parking to wait <span class="keyword">for</span> <<span class="number">0x000000076f5d4360</span>> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)</span><br><span class="line"> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:<span class="number">175</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:<span class="number">836</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:<span class="number">870</span>)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:<span class="number">1199</span>)</span><br><span class="line"> at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:<span class="number">209</span>)</span><br><span class="line"> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:<span class="number">285</span>)</span><br><span class="line"> at com.nicksxs.thread_dump_demo.ThreadDumpDemoApplication$<span class="number">1.</span>run(ThreadDumpDemoApplication.java:<span class="number">22</span>)</span><br><span class="line"></span><br><span class="line">Found <span class="number">1</span> deadlock.</span><br></pre></td></tr></table></figure>
|
|
<p>前面的信息其实上次就看过了，后面就可以发现有个死锁了，<br><img data-src="https://img.nicksxs.me/uPic/a1I2vj.png"><br>上面比较长，把主要的截出来，就是这边的，这点就很强大了。</p>
|
|
<h3 id="jmap"><a href="#jmap" class="headerlink" title="jmap"></a>jmap</h3><p>惯例还是看一下帮助信息<br><img data-src="https://img.nicksxs.me/uPic/dBiPqS.png"><br>这个相对命令比较多，不过因为现在 dump 下来我们可能会用文件模式，然后将文件下载下来使用 mat 进行分析，所以可以使用<br><code>jmap -dump:live,format=b,file=heap.bin <pid></code><br>命令照着上面看的就是打印活着的对象，然后以二进制格式，文件名叫 heap.bin 然后最后就是进程 id，打印出来以后可以用 mat 打开<br><img data-src="https://img.nicksxs.me/uPic/NRWowU.png"><br>这样就可以很清晰的看到应用里的各种信息，jmap 直接在命令中还可以看很多信息，比如使用<code>jmap -histo <pid></code>打印对象的实例数和对象占用的内存<br><img data-src="https://img.nicksxs.me/uPic/1GTBrY.png"><br><code>jmap -finalizerinfo <pid></code> 打印正在等候回收的对象<br><img data-src="https://img.nicksxs.me/uPic/Tb8uHP.png"></p>
|
|
<h4 id="小tips"><a href="#小tips" class="headerlink" title="小tips"></a>小tips</h4><p>对于一些应用内存已经占满了，jstack 和 jmap 可能会连不上的情况，可以使用<code>-F</code>参数强制打印线程或者 dump 文件，但是要注意这两者使用的用户必须与 java 进程启动用户一致，并且使用的 jdk 也要一致</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>工具</category>
|
|
<category>Thread dump</category>
|
|
<category>问题排查</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>JPS</tag>
|
|
<tag>JStack</tag>
|
|
<tag>JMap</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 RocketMQ 的 Broker 源码</title>
|
|
<url>/2020/07/19/%E8%81%8A%E8%81%8A-RocketMQ-%E7%9A%84-Broker-%E6%BA%90%E7%A0%81/</url>
|
|
<content><![CDATA[<p>broker 的启动形式有点类似于 NameServer，都是服务类型的，跟 Consumer 差别比较大，</p>
|
|
<p>首先是org.apache.rocketmq.broker.BrokerStartup中的 main 函数，org.apache.rocketmq.broker.BrokerStartup#createBrokerController基本就是读取参数，这里差点把最核心的初始化给漏了，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">final</span> <span class="type">BrokerController</span> <span class="variable">controller</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">BrokerController</span>(</span><br><span class="line"> brokerConfig,</span><br><span class="line"> nettyServerConfig,</span><br><span class="line"> nettyClientConfig,</span><br><span class="line"> messageStoreConfig);</span><br><span class="line"> <span class="comment">// remember all configs to prevent discard</span></span><br><span class="line"> controller.getConfiguration().registerConfig(properties);</span><br><span class="line"></span><br><span class="line"> <span class="type">boolean</span> <span class="variable">initResult</span> <span class="operator">=</span> controller.initialize();</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>前面是以 broker 配置，netty 的服务端和客户端配置，以及消息存储配置在实例化 BrokerController，然后就是初始化了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">initialize</span><span class="params">()</span> <span class="keyword">throws</span> CloneNotSupportedException {</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">result</span> <span class="operator">=</span> <span class="built_in">this</span>.topicConfigManager.load();</span><br><span class="line"></span><br><span class="line"> result = result && <span class="built_in">this</span>.consumerOffsetManager.load();</span><br><span class="line"> result = result && <span class="built_in">this</span>.subscriptionGroupManager.load();</span><br><span class="line"> result = result && <span class="built_in">this</span>.consumerFilterManager.load();</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>前面这些就是各个配置的 load 了，然后是个我认为比较重要的部分messageStore 的实例化，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">if</span> (result) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="built_in">this</span>.messageStore =</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">DefaultMessageStore</span>(<span class="built_in">this</span>.messageStoreConfig, <span class="built_in">this</span>.brokerStatsManager, <span class="built_in">this</span>.messageArrivingListener,</span><br><span class="line"> <span class="built_in">this</span>.brokerConfig);</span><br><span class="line"> <span class="keyword">if</span> (messageStoreConfig.isEnableDLegerCommitLog()) {</span><br><span class="line"> <span class="type">DLedgerRoleChangeHandler</span> <span class="variable">roleChangeHandler</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">DLedgerRoleChangeHandler</span>(<span class="built_in">this</span>, (DefaultMessageStore) messageStore);</span><br><span class="line"> ((DLedgerCommitLog)((DefaultMessageStore) messageStore).getCommitLog()).getdLedgerServer().getdLedgerLeaderElector().addRoleChangeHandler(roleChangeHandler);</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">this</span>.brokerStats = <span class="keyword">new</span> <span class="title class_">BrokerStats</span>((DefaultMessageStore) <span class="built_in">this</span>.messageStore);</span><br><span class="line"> <span class="comment">//load plugin</span></span><br><span class="line"> <span class="type">MessageStorePluginContext</span> <span class="variable">context</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">MessageStorePluginContext</span>(messageStoreConfig, brokerStatsManager, messageArrivingListener, brokerConfig);</span><br><span class="line"> <span class="built_in">this</span>.messageStore = MessageStoreFactory.build(context, <span class="built_in">this</span>.messageStore);</span><br><span class="line"> <span class="built_in">this</span>.messageStore.getDispatcherList().addFirst(<span class="keyword">new</span> <span class="title class_">CommitLogDispatcherCalcBitMap</span>(<span class="built_in">this</span>.brokerConfig, <span class="built_in">this</span>.consumerFilterManager));</span><br><span class="line"> } <span class="keyword">catch</span> (IOException e) {</span><br><span class="line"> result = <span class="literal">false</span>;</span><br><span class="line"> log.error(<span class="string">"Failed to initialize"</span>, e);</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">result = result && <span class="built_in">this</span>.messageStore.load();</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>先是实例化，实例化构造函数里的代码比较重要，重点看一下</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">DefaultMessageStore</span><span class="params">(<span class="keyword">final</span> MessageStoreConfig messageStoreConfig, <span class="keyword">final</span> BrokerStatsManager brokerStatsManager,</span></span><br><span class="line"><span class="params"> <span class="keyword">final</span> MessageArrivingListener messageArrivingListener, <span class="keyword">final</span> BrokerConfig brokerConfig)</span> <span class="keyword">throws</span> IOException {</span><br><span class="line"> <span class="built_in">this</span>.messageArrivingListener = messageArrivingListener;</span><br><span class="line"> <span class="built_in">this</span>.brokerConfig = brokerConfig;</span><br><span class="line"> <span class="built_in">this</span>.messageStoreConfig = messageStoreConfig;</span><br><span class="line"> <span class="built_in">this</span>.brokerStatsManager = brokerStatsManager;</span><br><span class="line"> <span class="built_in">this</span>.allocateMappedFileService = <span class="keyword">new</span> <span class="title class_">AllocateMappedFileService</span>(<span class="built_in">this</span>);</span><br><span class="line"> <span class="keyword">if</span> (messageStoreConfig.isEnableDLegerCommitLog()) {</span><br><span class="line"> <span class="built_in">this</span>.commitLog = <span class="keyword">new</span> <span class="title class_">DLedgerCommitLog</span>(<span class="built_in">this</span>);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="built_in">this</span>.commitLog = <span class="keyword">new</span> <span class="title class_">CommitLog</span>(<span class="built_in">this</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">this</span>.consumeQueueTable = <span class="keyword">new</span> <span class="title class_">ConcurrentHashMap</span><>(<span class="number">32</span>);</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.flushConsumeQueueService = <span class="keyword">new</span> <span class="title class_">FlushConsumeQueueService</span>();</span><br><span class="line"> <span class="built_in">this</span>.cleanCommitLogService = <span class="keyword">new</span> <span class="title class_">CleanCommitLogService</span>();</span><br><span class="line"> <span class="built_in">this</span>.cleanConsumeQueueService = <span class="keyword">new</span> <span class="title class_">CleanConsumeQueueService</span>();</span><br><span class="line"> <span class="built_in">this</span>.storeStatsService = <span class="keyword">new</span> <span class="title class_">StoreStatsService</span>();</span><br><span class="line"> <span class="built_in">this</span>.indexService = <span class="keyword">new</span> <span class="title class_">IndexService</span>(<span class="built_in">this</span>);</span><br><span class="line"> <span class="keyword">if</span> (!messageStoreConfig.isEnableDLegerCommitLog()) {</span><br><span class="line"> <span class="built_in">this</span>.haService = <span class="keyword">new</span> <span class="title class_">HAService</span>(<span class="built_in">this</span>);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="built_in">this</span>.haService = <span class="literal">null</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">this</span>.reputMessageService = <span class="keyword">new</span> <span class="title class_">ReputMessageService</span>();</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.scheduleMessageService = <span class="keyword">new</span> <span class="title class_">ScheduleMessageService</span>(<span class="built_in">this</span>);</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.transientStorePool = <span class="keyword">new</span> <span class="title class_">TransientStorePool</span>(messageStoreConfig);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (messageStoreConfig.isTransientStorePoolEnable()) {</span><br><span class="line"> <span class="built_in">this</span>.transientStorePool.init();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.allocateMappedFileService.start();</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.indexService.start();</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.dispatcherList = <span class="keyword">new</span> <span class="title class_">LinkedList</span><>();</span><br><span class="line"> <span class="built_in">this</span>.dispatcherList.addLast(<span class="keyword">new</span> <span class="title class_">CommitLogDispatcherBuildConsumeQueue</span>());</span><br><span class="line"> <span class="built_in">this</span>.dispatcherList.addLast(<span class="keyword">new</span> <span class="title class_">CommitLogDispatcherBuildIndex</span>());</span><br><span class="line"></span><br><span class="line"> <span class="type">File</span> <span class="variable">file</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">File</span>(StorePathConfigHelper.getLockFile(messageStoreConfig.getStorePathRootDir()));</span><br><span class="line"> MappedFile.ensureDirOK(file.getParent());</span><br><span class="line"> lockFile = <span class="keyword">new</span> <span class="title class_">RandomAccessFile</span>(file, <span class="string">"rw"</span>);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这里面有很多类，不过先把从构造函数里传进来的忽略下，接下来就是 <code>AllocateMappedFileService</code> 这个service，前面看过文章的可能会根据上面的代码猜到，这也是个 ServiceThread，如果是对RocketMQ 有所了解的可能从名字可以看出这个类是关于 RocketMQ 消息怎么落盘的，当需要创建MappedFile时（在MapedFileQueue.getLastMapedFile方法中），向该线程的requestQueue队列中放入AllocateRequest请求对象，该线程会在后台监听该队列，并在后台创建MapedFile对象，即同时创建了物理文件。然后是创建了 IndexService 服务线程，用来给创建索引；还有是FlushConsumeQueueService是将ConsumeQueue 刷入磁盘；CleanCommitLogService用来清理过期的 CommitLog，默认是 72 小时以上；CleanConsumeQueueService是将小于最新的 CommitLog 偏移量的 ConsumeQueue 清理掉；StoreStatsService是储存统计服务；HAService用于CommitLog 的主备同步；ScheduleMessageService用于定时消息；还有就是这个ReputMessageService非常重要，就是由它实现了将 CommitLog 以 topic+queue 纬度构建 ConsumeQueue，后面TransientStorePool是异步刷盘时的存储buffer，也可以从后面的判断中看出来</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">isTransientStorePoolEnable</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> transientStorePoolEnable && FlushDiskType.ASYNC_FLUSH == getFlushDiskType()</span><br><span class="line"> && BrokerRole.SLAVE != getBrokerRole();</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>再然后就是启动两个服务线程，dispatcherList是为CommitLog文件转发请求，差不多这个初始化就这些内容。</p>
|
|
<p>然后回到外层，下面是主备切换的配置，然后是数据统计，接着是存储插件加载，然后是往转发器链表里再加一个过滤器</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">if</span> (messageStoreConfig.isEnableDLegerCommitLog()) {</span><br><span class="line"> <span class="type">DLedgerRoleChangeHandler</span> <span class="variable">roleChangeHandler</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">DLedgerRoleChangeHandler</span>(<span class="built_in">this</span>, (DefaultMessageStore) messageStore);</span><br><span class="line"> ((DLedgerCommitLog)((DefaultMessageStore) messageStore).getCommitLog()).getdLedgerServer().getdLedgerLeaderElector().addRoleChangeHandler(roleChangeHandler);</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">this</span>.brokerStats = <span class="keyword">new</span> <span class="title class_">BrokerStats</span>((DefaultMessageStore) <span class="built_in">this</span>.messageStore);</span><br><span class="line"> <span class="comment">//load plugin</span></span><br><span class="line"> <span class="type">MessageStorePluginContext</span> <span class="variable">context</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">MessageStorePluginContext</span>(messageStoreConfig, brokerStatsManager, messageArrivingListener, brokerConfig);</span><br><span class="line"> <span class="built_in">this</span>.messageStore = MessageStoreFactory.build(context, <span class="built_in">this</span>.messageStore);</span><br><span class="line"> <span class="built_in">this</span>.messageStore.getDispatcherList().addFirst(<span class="keyword">new</span> <span class="title class_">CommitLogDispatcherCalcBitMap</span>(<span class="built_in">this</span>.brokerConfig, <span class="built_in">this</span>.consumerFilterManager));</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>接下来就是org.apache.rocketmq.store.MessageStore#load的过程了，</p>
|
|
<ol>
|
|
<li>调用ScheduleMessageService.load方法，初始化延迟级别列表。将这些级别（”1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h”）的延时存入延迟级别delayLevelTable：ConcurrentHashMap<Integer /* level */, Long/* delay timeMillis */>变量中，例如1s的kv值为1:1000,5s的kv值为2:5000，key值依次类推；每个延迟级别即为一个队列。</li>
|
|
</ol>
|
|
<p>2)调用CommitLog.load方法，在此方法中调用MapedFileQueue.load方法，将$HOME /store/commitlog目录下的所有文件加载到MapedFileQueue的List<MapedFile>变量中；</p>
|
|
<p>3)调用DefaultMessageStore.loadConsumeQueue方法加载consumequeue文件数据到DefaultMessageStore.consumeQueueTable集合中。</p>
|
|
<p>初始化StoreCheckPoint对象，加载$HOME/store/checkpoint文件，该文件记录三个字段值，分别是物理队列消息时间戳、逻辑队列消息时间戳、索引队列消息时间戳。</p>
|
|
<p>调用IndexService.load方法加载$HOME/store/index目录下的文件。对该目录下的每个文件初始化一个IndexFile对象。然后调用IndexFile对象的load方法将IndexHeader加载到对象的变量中；再根据检查是否存在abort文件，若有存在abort文件，则表示Broker表示上次是异常退出的，则检查checkpoint的indexMsgTimestamp字段值是否小于IndexHeader的endTimestamp值，indexMsgTimestamp值表示最后刷盘的时间，若小于则表示在最后刷盘之后在该文件中还创建了索引，则要删除该Index文件，否则将该IndexFile对象放入indexFileList:ArrayList<IndexFile>索引文件集合中。</p>
|
|
<p>然后调用org.apache.rocketmq.store.DefaultMessageStore#recover恢复，前面有根据<code>boolean lastExitOK = !this.isTempFileExist();</code>临时文件是否存在来判断上一次是否正常退出，根据这个状态来选择什么恢复策略</p>
|
|
<p>接下去是初始化 Netty 服务端，初始化发送消息线程池（sendMessageExecutor）、拉取消息线程池（pullMessageExecutor）、管理Broker线程池（adminBrokerExecutor）、客户端管理线程池（clientManageExecutor），注册事件处理器，包括发送消息事件处理器（SendMessageProcessor）、拉取消息事件处理器、查询消息事件处理器（QueryMessageProcessor，包括客户端的心跳事件、注销事件、获取消费者列表事件、更新更新和查询消费进度consumerOffset）、客户端管理事件处理器（ClientManageProcessor）、结束事务处理器（EndTransactionProcessor）、默认事件处理器（AdminBrokerProcessor），然后是定时任务</p>
|
|
<p><code>BrokerController.this.getBrokerStats().record();</code> 记录 Broker 状态</p>
|
|
<p><code>BrokerController.this.consumerOffsetManager.persist();</code> 持久化consumerOffset</p>
|
|
<p><code>BrokerController.this.consumerFilterManager.persist();</code>持久化consumerFilter</p>
|
|
<p><code>BrokerController.this.protectBroker();</code> 保护 broker，消费慢，不让继续投递</p>
|
|
<p><code>BrokerController.this.printWaterMark();</code> 打印水位</p>
|
|
<p><code>log.info("dispatch behind commit log {} bytes", BrokerController.this.getMessageStore().dispatchBehindBytes());</code> 检查落后程度</p>
|
|
<p><code>BrokerController.this.brokerOuterAPI.fetchNameServerAddr();</code> 定时获取 nameserver</p>
|
|
<p><code>BrokerController.this.printMasterAndSlaveDiff();</code> 打印主从不一致</p>
|
|
<p>然后是 tsl，初始化事务消息，初始化 RPCHook</p>
|
|
<p>请把害怕打到公屏上🤦♂️，从线程池名字和调用的方法应该可以看出大部分的用途</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="built_in">this</span>.remotingServer = <span class="keyword">new</span> <span class="title class_">NettyRemotingServer</span>(<span class="built_in">this</span>.nettyServerConfig, <span class="built_in">this</span>.clientHousekeepingService);</span><br><span class="line"> <span class="type">NettyServerConfig</span> <span class="variable">fastConfig</span> <span class="operator">=</span> (NettyServerConfig) <span class="built_in">this</span>.nettyServerConfig.clone();</span><br><span class="line"> fastConfig.setListenPort(nettyServerConfig.getListenPort() - <span class="number">2</span>);</span><br><span class="line"> <span class="built_in">this</span>.fastRemotingServer = <span class="keyword">new</span> <span class="title class_">NettyRemotingServer</span>(fastConfig, <span class="built_in">this</span>.clientHousekeepingService);</span><br><span class="line"> <span class="built_in">this</span>.sendMessageExecutor = <span class="keyword">new</span> <span class="title class_">BrokerFixedThreadPoolExecutor</span>(</span><br><span class="line"> <span class="built_in">this</span>.brokerConfig.getSendMessageThreadPoolNums(),</span><br><span class="line"> <span class="built_in">this</span>.brokerConfig.getSendMessageThreadPoolNums(),</span><br><span class="line"> <span class="number">1000</span> * <span class="number">60</span>,</span><br><span class="line"> TimeUnit.MILLISECONDS,</span><br><span class="line"> <span class="built_in">this</span>.sendThreadPoolQueue,</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">ThreadFactoryImpl</span>(<span class="string">"SendMessageThread_"</span>));</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.pullMessageExecutor = <span class="keyword">new</span> <span class="title class_">BrokerFixedThreadPoolExecutor</span>(</span><br><span class="line"> <span class="built_in">this</span>.brokerConfig.getPullMessageThreadPoolNums(),</span><br><span class="line"> <span class="built_in">this</span>.brokerConfig.getPullMessageThreadPoolNums(),</span><br><span class="line"> <span class="number">1000</span> * <span class="number">60</span>,</span><br><span class="line"> TimeUnit.MILLISECONDS,</span><br><span class="line"> <span class="built_in">this</span>.pullThreadPoolQueue,</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">ThreadFactoryImpl</span>(<span class="string">"PullMessageThread_"</span>));</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.replyMessageExecutor = <span class="keyword">new</span> <span class="title class_">BrokerFixedThreadPoolExecutor</span>(</span><br><span class="line"> <span class="built_in">this</span>.brokerConfig.getProcessReplyMessageThreadPoolNums(),</span><br><span class="line"> <span class="built_in">this</span>.brokerConfig.getProcessReplyMessageThreadPoolNums(),</span><br><span class="line"> <span class="number">1000</span> * <span class="number">60</span>,</span><br><span class="line"> TimeUnit.MILLISECONDS,</span><br><span class="line"> <span class="built_in">this</span>.replyThreadPoolQueue,</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">ThreadFactoryImpl</span>(<span class="string">"ProcessReplyMessageThread_"</span>));</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.queryMessageExecutor = <span class="keyword">new</span> <span class="title class_">BrokerFixedThreadPoolExecutor</span>(</span><br><span class="line"> <span class="built_in">this</span>.brokerConfig.getQueryMessageThreadPoolNums(),</span><br><span class="line"> <span class="built_in">this</span>.brokerConfig.getQueryMessageThreadPoolNums(),</span><br><span class="line"> <span class="number">1000</span> * <span class="number">60</span>,</span><br><span class="line"> TimeUnit.MILLISECONDS,</span><br><span class="line"> <span class="built_in">this</span>.queryThreadPoolQueue,</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">ThreadFactoryImpl</span>(<span class="string">"QueryMessageThread_"</span>));</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.adminBrokerExecutor =</span><br><span class="line"> Executors.newFixedThreadPool(<span class="built_in">this</span>.brokerConfig.getAdminBrokerThreadPoolNums(), <span class="keyword">new</span> <span class="title class_">ThreadFactoryImpl</span>(</span><br><span class="line"> <span class="string">"AdminBrokerThread_"</span>));</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.clientManageExecutor = <span class="keyword">new</span> <span class="title class_">ThreadPoolExecutor</span>(</span><br><span class="line"> <span class="built_in">this</span>.brokerConfig.getClientManageThreadPoolNums(),</span><br><span class="line"> <span class="built_in">this</span>.brokerConfig.getClientManageThreadPoolNums(),</span><br><span class="line"> <span class="number">1000</span> * <span class="number">60</span>,</span><br><span class="line"> TimeUnit.MILLISECONDS,</span><br><span class="line"> <span class="built_in">this</span>.clientManagerThreadPoolQueue,</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">ThreadFactoryImpl</span>(<span class="string">"ClientManageThread_"</span>));</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.heartbeatExecutor = <span class="keyword">new</span> <span class="title class_">BrokerFixedThreadPoolExecutor</span>(</span><br><span class="line"> <span class="built_in">this</span>.brokerConfig.getHeartbeatThreadPoolNums(),</span><br><span class="line"> <span class="built_in">this</span>.brokerConfig.getHeartbeatThreadPoolNums(),</span><br><span class="line"> <span class="number">1000</span> * <span class="number">60</span>,</span><br><span class="line"> TimeUnit.MILLISECONDS,</span><br><span class="line"> <span class="built_in">this</span>.heartbeatThreadPoolQueue,</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">ThreadFactoryImpl</span>(<span class="string">"HeartbeatThread_"</span>, <span class="literal">true</span>));</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.endTransactionExecutor = <span class="keyword">new</span> <span class="title class_">BrokerFixedThreadPoolExecutor</span>(</span><br><span class="line"> <span class="built_in">this</span>.brokerConfig.getEndTransactionThreadPoolNums(),</span><br><span class="line"> <span class="built_in">this</span>.brokerConfig.getEndTransactionThreadPoolNums(),</span><br><span class="line"> <span class="number">1000</span> * <span class="number">60</span>,</span><br><span class="line"> TimeUnit.MILLISECONDS,</span><br><span class="line"> <span class="built_in">this</span>.endTransactionThreadPoolQueue,</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">ThreadFactoryImpl</span>(<span class="string">"EndTransactionThread_"</span>));</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.consumerManageExecutor =</span><br><span class="line"> Executors.newFixedThreadPool(<span class="built_in">this</span>.brokerConfig.getConsumerManageThreadPoolNums(), <span class="keyword">new</span> <span class="title class_">ThreadFactoryImpl</span>(</span><br><span class="line"> <span class="string">"ConsumerManageThread_"</span>));</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.registerProcessor();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">final</span> <span class="type">long</span> <span class="variable">initialDelay</span> <span class="operator">=</span> UtilAll.computeNextMorningTimeMillis() - System.currentTimeMillis();</span><br><span class="line"> <span class="keyword">final</span> <span class="type">long</span> <span class="variable">period</span> <span class="operator">=</span> <span class="number">1000</span> * <span class="number">60</span> * <span class="number">60</span> * <span class="number">24</span>;</span><br><span class="line"> <span class="built_in">this</span>.scheduledExecutorService.scheduleAtFixedRate(<span class="keyword">new</span> <span class="title class_">Runnable</span>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> BrokerController.<span class="built_in">this</span>.getBrokerStats().record();</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> log.error(<span class="string">"schedule record error."</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }, initialDelay, period, TimeUnit.MILLISECONDS);</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.scheduledExecutorService.scheduleAtFixedRate(<span class="keyword">new</span> <span class="title class_">Runnable</span>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> BrokerController.<span class="built_in">this</span>.consumerOffsetManager.persist();</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> log.error(<span class="string">"schedule persist consumerOffset error."</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }, <span class="number">1000</span> * <span class="number">10</span>, <span class="built_in">this</span>.brokerConfig.getFlushConsumerOffsetInterval(), TimeUnit.MILLISECONDS);</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.scheduledExecutorService.scheduleAtFixedRate(<span class="keyword">new</span> <span class="title class_">Runnable</span>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> BrokerController.<span class="built_in">this</span>.consumerFilterManager.persist();</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> log.error(<span class="string">"schedule persist consumer filter error."</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }, <span class="number">1000</span> * <span class="number">10</span>, <span class="number">1000</span> * <span class="number">10</span>, TimeUnit.MILLISECONDS);</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.scheduledExecutorService.scheduleAtFixedRate(<span class="keyword">new</span> <span class="title class_">Runnable</span>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> BrokerController.<span class="built_in">this</span>.protectBroker();</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> log.error(<span class="string">"protectBroker error."</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }, <span class="number">3</span>, <span class="number">3</span>, TimeUnit.MINUTES);</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.scheduledExecutorService.scheduleAtFixedRate(<span class="keyword">new</span> <span class="title class_">Runnable</span>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> BrokerController.<span class="built_in">this</span>.printWaterMark();</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> log.error(<span class="string">"printWaterMark error."</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }, <span class="number">10</span>, <span class="number">1</span>, TimeUnit.SECONDS);</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.scheduledExecutorService.scheduleAtFixedRate(<span class="keyword">new</span> <span class="title class_">Runnable</span>() {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> log.info(<span class="string">"dispatch behind commit log {} bytes"</span>, BrokerController.<span class="built_in">this</span>.getMessageStore().dispatchBehindBytes());</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> log.error(<span class="string">"schedule dispatchBehindBytes error."</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }, <span class="number">1000</span> * <span class="number">10</span>, <span class="number">1000</span> * <span class="number">60</span>, TimeUnit.MILLISECONDS);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.brokerConfig.getNamesrvAddr() != <span class="literal">null</span>) {</span><br><span class="line"> <span class="built_in">this</span>.brokerOuterAPI.updateNameServerAddressList(<span class="built_in">this</span>.brokerConfig.getNamesrvAddr());</span><br><span class="line"> log.info(<span class="string">"Set user specified name server address: {}"</span>, <span class="built_in">this</span>.brokerConfig.getNamesrvAddr());</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (<span class="built_in">this</span>.brokerConfig.isFetchNamesrvAddrByAddressServer()) {</span><br><span class="line"> <span class="built_in">this</span>.scheduledExecutorService.scheduleAtFixedRate(<span class="keyword">new</span> <span class="title class_">Runnable</span>() {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> BrokerController.<span class="built_in">this</span>.brokerOuterAPI.fetchNameServerAddr();</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> log.error(<span class="string">"ScheduledTask fetchNameServerAddr exception"</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }, <span class="number">1000</span> * <span class="number">10</span>, <span class="number">1000</span> * <span class="number">60</span> * <span class="number">2</span>, TimeUnit.MILLISECONDS);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!messageStoreConfig.isEnableDLegerCommitLog()) {</span><br><span class="line"> <span class="keyword">if</span> (BrokerRole.SLAVE == <span class="built_in">this</span>.messageStoreConfig.getBrokerRole()) {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.messageStoreConfig.getHaMasterAddress() != <span class="literal">null</span> && <span class="built_in">this</span>.messageStoreConfig.getHaMasterAddress().length() >= <span class="number">6</span>) {</span><br><span class="line"> <span class="built_in">this</span>.messageStore.updateHaMasterAddress(<span class="built_in">this</span>.messageStoreConfig.getHaMasterAddress());</span><br><span class="line"> <span class="built_in">this</span>.updateMasterHAServerAddrPeriodically = <span class="literal">false</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="built_in">this</span>.updateMasterHAServerAddrPeriodically = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="built_in">this</span>.scheduledExecutorService.scheduleAtFixedRate(<span class="keyword">new</span> <span class="title class_">Runnable</span>() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> BrokerController.<span class="built_in">this</span>.printMasterAndSlaveDiff();</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> log.error(<span class="string">"schedule printMasterAndSlaveDiff error."</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }, <span class="number">1000</span> * <span class="number">10</span>, <span class="number">1000</span> * <span class="number">60</span>, TimeUnit.MILLISECONDS);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (TlsSystemConfig.tlsMode != TlsMode.DISABLED) {</span><br><span class="line"> <span class="comment">// Register a listener to reload SslContext</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> fileWatchService = <span class="keyword">new</span> <span class="title class_">FileWatchService</span>(</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">String</span>[] {</span><br><span class="line"> TlsSystemConfig.tlsServerCertPath,</span><br><span class="line"> TlsSystemConfig.tlsServerKeyPath,</span><br><span class="line"> TlsSystemConfig.tlsServerTrustCertPath</span><br><span class="line"> },</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">FileWatchService</span>.Listener() {</span><br><span class="line"> <span class="type">boolean</span> certChanged, keyChanged = <span class="literal">false</span>;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">onChanged</span><span class="params">(String path)</span> {</span><br><span class="line"> <span class="keyword">if</span> (path.equals(TlsSystemConfig.tlsServerTrustCertPath)) {</span><br><span class="line"> log.info(<span class="string">"The trust certificate changed, reload the ssl context"</span>);</span><br><span class="line"> reloadServerSslContext();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (path.equals(TlsSystemConfig.tlsServerCertPath)) {</span><br><span class="line"> certChanged = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (path.equals(TlsSystemConfig.tlsServerKeyPath)) {</span><br><span class="line"> keyChanged = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (certChanged && keyChanged) {</span><br><span class="line"> log.info(<span class="string">"The certificate and private key changed, reload the ssl context"</span>);</span><br><span class="line"> certChanged = keyChanged = <span class="literal">false</span>;</span><br><span class="line"> reloadServerSslContext();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">reloadServerSslContext</span><span class="params">()</span> {</span><br><span class="line"> ((NettyRemotingServer) remotingServer).loadSslContext();</span><br><span class="line"> ((NettyRemotingServer) fastRemotingServer).loadSslContext();</span><br><span class="line"> }</span><br><span class="line"> });</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> log.warn(<span class="string">"FileWatchService created error, can't load the certificate dynamically"</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> initialTransaction();</span><br><span class="line"> initialAcl();</span><br><span class="line"> initialRpcHooks();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> result;</span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
|
|
<p>Broker 启动过程</p>
|
|
<p>贴代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">start</span><span class="params">()</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.messageStore != <span class="literal">null</span>) {</span><br><span class="line"> <span class="built_in">this</span>.messageStore.start();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.remotingServer != <span class="literal">null</span>) {</span><br><span class="line"> <span class="built_in">this</span>.remotingServer.start();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.fastRemotingServer != <span class="literal">null</span>) {</span><br><span class="line"> <span class="built_in">this</span>.fastRemotingServer.start();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.fileWatchService != <span class="literal">null</span>) {</span><br><span class="line"> <span class="built_in">this</span>.fileWatchService.start();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.brokerOuterAPI != <span class="literal">null</span>) {</span><br><span class="line"> <span class="built_in">this</span>.brokerOuterAPI.start();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.pullRequestHoldService != <span class="literal">null</span>) {</span><br><span class="line"> <span class="built_in">this</span>.pullRequestHoldService.start();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.clientHousekeepingService != <span class="literal">null</span>) {</span><br><span class="line"> <span class="built_in">this</span>.clientHousekeepingService.start();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.filterServerManager != <span class="literal">null</span>) {</span><br><span class="line"> <span class="built_in">this</span>.filterServerManager.start();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!messageStoreConfig.isEnableDLegerCommitLog()) {</span><br><span class="line"> startProcessorByHa(messageStoreConfig.getBrokerRole());</span><br><span class="line"> handleSlaveSynchronize(messageStoreConfig.getBrokerRole());</span><br><span class="line"> <span class="built_in">this</span>.registerBrokerAll(<span class="literal">true</span>, <span class="literal">false</span>, <span class="literal">true</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.scheduledExecutorService.scheduleAtFixedRate(<span class="keyword">new</span> <span class="title class_">Runnable</span>() {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> BrokerController.<span class="built_in">this</span>.registerBrokerAll(<span class="literal">true</span>, <span class="literal">false</span>, brokerConfig.isForceRegister());</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> log.error(<span class="string">"registerBrokerAll Exception"</span>, e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }, <span class="number">1000</span> * <span class="number">10</span>, Math.max(<span class="number">10000</span>, Math.min(brokerConfig.getRegisterNameServerPeriod(), <span class="number">60000</span>)), TimeUnit.MILLISECONDS);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.brokerStatsManager != <span class="literal">null</span>) {</span><br><span class="line"> <span class="built_in">this</span>.brokerStatsManager.start();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">this</span>.brokerFastFailure != <span class="literal">null</span>) {</span><br><span class="line"> <span class="built_in">this</span>.brokerFastFailure.start();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>首先是启动messageStore，调用 start 方法，这里面又调用了一些代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">start</span><span class="params">()</span> <span class="keyword">throws</span> Exception {</span><br><span class="line"></span><br><span class="line"> lock = lockFile.getChannel().tryLock(<span class="number">0</span>, <span class="number">1</span>, <span class="literal">false</span>);</span><br><span class="line"> <span class="keyword">if</span> (lock == <span class="literal">null</span> || lock.isShared() || !lock.isValid()) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">RuntimeException</span>(<span class="string">"Lock failed,MQ already started"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> lockFile.getChannel().write(ByteBuffer.wrap(<span class="string">"lock"</span>.getBytes()));</span><br><span class="line"> lockFile.getChannel().force(<span class="literal">true</span>);</span><br><span class="line"> {</span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 1. Make sure the fast-forward messages to be truncated during the recovering according to the max physical offset of the commitlog;</span></span><br><span class="line"><span class="comment"> * 2. DLedger committedPos may be missing, so the maxPhysicalPosInLogicQueue maybe bigger that maxOffset returned by DLedgerCommitLog, just let it go;</span></span><br><span class="line"><span class="comment"> * 3. Calculate the reput offset according to the consume queue;</span></span><br><span class="line"><span class="comment"> * 4. Make sure the fall-behind messages to be dispatched before starting the commitlog, especially when the broker role are automatically changed.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="type">long</span> <span class="variable">maxPhysicalPosInLogicQueue</span> <span class="operator">=</span> commitLog.getMinOffset();</span><br><span class="line"> <span class="keyword">for</span> (ConcurrentMap<Integer, ConsumeQueue> maps : <span class="built_in">this</span>.consumeQueueTable.values()) {</span><br><span class="line"> <span class="keyword">for</span> (ConsumeQueue logic : maps.values()) {</span><br><span class="line"> <span class="keyword">if</span> (logic.getMaxPhysicOffset() > maxPhysicalPosInLogicQueue) {</span><br><span class="line"> maxPhysicalPosInLogicQueue = logic.getMaxPhysicOffset();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (maxPhysicalPosInLogicQueue < <span class="number">0</span>) {</span><br><span class="line"> maxPhysicalPosInLogicQueue = <span class="number">0</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (maxPhysicalPosInLogicQueue < <span class="built_in">this</span>.commitLog.getMinOffset()) {</span><br><span class="line"> maxPhysicalPosInLogicQueue = <span class="built_in">this</span>.commitLog.getMinOffset();</span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * This happens in following conditions:</span></span><br><span class="line"><span class="comment"> * 1. If someone removes all the consumequeue files or the disk get damaged.</span></span><br><span class="line"><span class="comment"> * 2. Launch a new broker, and copy the commitlog from other brokers.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * All the conditions has the same in common that the maxPhysicalPosInLogicQueue should be 0.</span></span><br><span class="line"><span class="comment"> * If the maxPhysicalPosInLogicQueue is gt 0, there maybe something wrong.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> log.warn(<span class="string">"[TooSmallCqOffset] maxPhysicalPosInLogicQueue={} clMinOffset={}"</span>, maxPhysicalPosInLogicQueue, <span class="built_in">this</span>.commitLog.getMinOffset());</span><br><span class="line"> }</span><br><span class="line"> log.info(<span class="string">"[SetReputOffset] maxPhysicalPosInLogicQueue={} clMinOffset={} clMaxOffset={} clConfirmedOffset={}"</span>,</span><br><span class="line"> maxPhysicalPosInLogicQueue, <span class="built_in">this</span>.commitLog.getMinOffset(), <span class="built_in">this</span>.commitLog.getMaxOffset(), <span class="built_in">this</span>.commitLog.getConfirmOffset());</span><br><span class="line"> <span class="built_in">this</span>.reputMessageService.setReputFromOffset(maxPhysicalPosInLogicQueue);</span><br><span class="line"> <span class="built_in">this</span>.reputMessageService.start();</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 1. Finish dispatching the messages fall behind, then to start other services.</span></span><br><span class="line"><span class="comment"> * 2. DLedger committedPos may be missing, so here just require dispatchBehindBytes <= 0</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">while</span> (<span class="literal">true</span>) {</span><br><span class="line"> <span class="keyword">if</span> (dispatchBehindBytes() <= <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> Thread.sleep(<span class="number">1000</span>);</span><br><span class="line"> log.info(<span class="string">"Try to finish doing reput the messages fall behind during the starting, reputOffset={} maxOffset={} behind={}"</span>, <span class="built_in">this</span>.reputMessageService.getReputFromOffset(), <span class="built_in">this</span>.getMaxPhyOffset(), <span class="built_in">this</span>.dispatchBehindBytes());</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">this</span>.recoverTopicQueueTable();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!messageStoreConfig.isEnableDLegerCommitLog()) {</span><br><span class="line"> <span class="built_in">this</span>.haService.start();</span><br><span class="line"> <span class="built_in">this</span>.handleScheduleMessageService(messageStoreConfig.getBrokerRole());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.flushConsumeQueueService.start();</span><br><span class="line"> <span class="built_in">this</span>.commitLog.start();</span><br><span class="line"> <span class="built_in">this</span>.storeStatsService.start();</span><br><span class="line"></span><br><span class="line"> <span class="built_in">this</span>.createTempFile();</span><br><span class="line"> <span class="built_in">this</span>.addScheduleTask();</span><br><span class="line"> <span class="built_in">this</span>.shutdown = <span class="literal">false</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
|
|
|
|
|
|
<p>调用DefaultMessageStore.start方法启动DefaultMessageStore对象中的一些服务线程。</p>
|
|
<ol>
|
|
<li>启动ReputMessageService服务线程</li>
|
|
<li>启动FlushConsumeQueueService服务线程；</li>
|
|
<li>调用CommitLog.start方法，启动CommitLog对象中的FlushCommitLogService线程服务，若是同步刷盘（SYNC_FLUSH）则是启动GroupCommitService线程服务；若是异步刷盘（ASYNC_FLUSH）则是启动FlushRealTimeService线程服务；</li>
|
|
<li>启动StoreStatsService线程服务；</li>
|
|
<li>启动定时清理任务</li>
|
|
</ol>
|
|
<p>然后是启动ClientHousekeepingService的 netty 服务端和客户端，然后是启动fileWatchService证书服务，接着启动BrokerOuterAPI中的NettyRemotingClient，即建立与NameServer的链接，用于自身Broker与其他模块的RPC功能调用；包括获取NameServer的地址、注册Broker、注销Broker、获取Topic配置、获取消息进度信息、获取订阅关系等RPC功能，然后是PullRequestHoldService服务线程，这个就是实现长轮询的，然后启动管家ClientHousekeepingService服务，负责扫描不活跃的生产者，消费者和 filter，启动FilterServerManager 过滤器服务管理，然后启动定时任务调用org.apache.rocketmq.broker.BrokerController#registerBrokerAll向所有 nameserver 注册 broker，最后是按需开启org.apache.rocketmq.store.stats.BrokerStatsManager和org.apache.rocketmq.broker.latency.BrokerFastFailure，基本上启动过程就完成了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>MQ</category>
|
|
<category>RocketMQ</category>
|
|
<category>消息队列</category>
|
|
<category>RocketMQ</category>
|
|
<category>中间件</category>
|
|
<category>RocketMQ</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>MQ</tag>
|
|
<tag>消息队列</tag>
|
|
<tag>RocketMQ</tag>
|
|
<tag>削峰填谷</tag>
|
|
<tag>中间件</tag>
|
|
<tag>源码解析</tag>
|
|
<tag>Broker</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 Sharding-Jdbc 分库分表下的分页方案</title>
|
|
<url>/2022/01/09/%E8%81%8A%E8%81%8A-Sharding-Jdbc-%E5%88%86%E5%BA%93%E5%88%86%E8%A1%A8%E4%B8%8B%E7%9A%84%E5%88%86%E9%A1%B5%E6%96%B9%E6%A1%88/</url>
|
|
<content><![CDATA[<p>前面在聊 Sharding-Jdbc 的时候看到了一篇文章，关于一个分页的查询，一直比较直接的想法就是分库分表下的分页是非常不合理的，一般我们的实操方案都是分表加上 ES 搜索做分页，或者通过合表读写分离的方案，因为对于 sharding-jdbc 如果没有带分表键，查询基本都是只能在所有分表都执行一遍，然后再加上分页，基本上是分页越大后续的查询越耗资源，但是仔细的去想这个细节还是这次，就简单说说<br>首先就是我的分表结构</p>
|
|
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> `student_time_0` (</span><br><span class="line"> `id` <span class="type">int</span>(<span class="number">11</span>) unsigned <span class="keyword">NOT</span> <span class="keyword">NULL</span> AUTO_INCREMENT,</span><br><span class="line"> `user_id` <span class="type">int</span>(<span class="number">11</span>) <span class="keyword">NOT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `name` <span class="type">varchar</span>(<span class="number">200</span>) <span class="keyword">COLLATE</span> utf8_bin <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `age` tinyint(<span class="number">3</span>) unsigned <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `create_time` <span class="type">bigint</span>(<span class="number">20</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> <span class="keyword">PRIMARY</span> KEY (`id`)</span><br><span class="line">) ENGINE<span class="operator">=</span>InnoDB AUTO_INCREMENT<span class="operator">=</span><span class="number">674</span> <span class="keyword">DEFAULT</span> CHARSET<span class="operator">=</span>utf8 <span class="keyword">COLLATE</span><span class="operator">=</span>utf8_bin;</span><br></pre></td></tr></table></figure>
|
|
<p>有这样的三个表，<code>student_time_0</code>, <code>student_time_1</code>, <code>student_time_2</code>, 以 user_id 作为分表键，根据表数量取模作为分表依据<br>这里先构造点数据，</p>
|
|
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="keyword">insert</span> <span class="keyword">into</span> student_time (`name`, `user_id`, `age`, `create_time`) <span class="keyword">values</span> (?, ?, ?, ?)</span><br></pre></td></tr></table></figure>
|
|
<p>主要是为了保证 <code>create_time</code> 唯一比较好说明问题，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"><span class="keyword">try</span> (</span><br><span class="line"> <span class="type">Connection</span> <span class="variable">conn</span> <span class="operator">=</span> dataSource.getConnection();</span><br><span class="line"> <span class="type">PreparedStatement</span> <span class="variable">ps</span> <span class="operator">=</span> conn.prepareStatement(insertSql)) {</span><br><span class="line"> <span class="keyword">do</span> {</span><br><span class="line"> ps.setString(<span class="number">1</span>, localName + <span class="keyword">new</span> <span class="title class_">Random</span>().nextInt(<span class="number">100</span>));</span><br><span class="line"> ps.setLong(<span class="number">2</span>, <span class="number">10086L</span> + (<span class="keyword">new</span> <span class="title class_">Random</span>().nextInt(<span class="number">100</span>)));</span><br><span class="line"> ps.setInt(<span class="number">3</span>, <span class="number">18</span>);</span><br><span class="line"> ps.setLong(<span class="number">4</span>, <span class="keyword">new</span> <span class="title class_">Date</span>().getTime());</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="type">int</span> <span class="variable">result</span> <span class="operator">=</span> ps.executeUpdate();</span><br><span class="line"> LOGGER.info(<span class="string">"current execute result: {}"</span>, result);</span><br><span class="line"> Thread.sleep(<span class="keyword">new</span> <span class="title class_">Random</span>().nextInt(<span class="number">100</span>));</span><br><span class="line"> i++;</span><br><span class="line"> } <span class="keyword">while</span> (i <= <span class="number">2000</span>);</span><br></pre></td></tr></table></figure>
|
|
<p>三个表的数据分别是 673，678，650，说明符合预期了，各个表数据不一样，接下来比如我们想要做一个这样的分页查询</p>
|
|
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="keyword">select</span> <span class="operator">*</span> <span class="keyword">from</span> student_time <span class="keyword">ORDER</span> <span class="keyword">BY</span> create_time <span class="keyword">ASC</span> limit <span class="number">1000</span>, <span class="number">5</span>;</span><br></pre></td></tr></table></figure>
|
|
<p><code>student_time</code> 对于我们使用的 <code>sharding-jdbc</code> 来说当然是逻辑表，首先从一无所知去想这个查询如果我们自己来处理应该是怎么做，<br>首先是不是可以每个表都从 333 开始取 5 条数据，类似于下面的查询，然后进行 15 条的合并重排序获取前面的 5 条</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">select * from student_time_0 ORDER BY create_time ASC limit 333, 5;</span><br><span class="line">select * from student_time_1 ORDER BY create_time ASC limit 333, 5;</span><br><span class="line">select * from student_time_2 ORDER BY create_time ASC limit 333, 5;</span><br></pre></td></tr></table></figure>
|
|
<p>忽略前面 limit 差的 1，这个结果除非三个表的分布是绝对的均匀，否则结果肯定会出现一定的偏差，以为每个表的 333 这个位置对于其他表来说都不一定是一样的，这样对于最后整体的结果，就会出现偏差<br>因为一直在纠结怎么让这个更直观的表现出来，所以尝试画了个图<br><img data-src="https://img.nicksxs.me/uPic/Gj7TPq.png"><br>黑色的框代表我从每个表里按排序从 334 到 338 的 5 条数据， 他们在每个表里都是代表了各自正确的排序值，但是对于我们想要的其实是合表后的 1001，1005 这五条，然后我们假设总的排序值位于前 1000 的分布是第 0 个表是 320 条，第 1 个表是 340 条，第 2 个表是 340 条，那么可以明显地看出来我这么查的结果简单合并肯定是不对的。<br>那么 sharding-jdbc 是如何保证这个结果的呢，其实就是我在每个表里都查分页偏移量和分页大小那么多的数据，在我这个例子里就是对于 0，1，2 三个分表每个都查 1005 条数据，即使我的数据不平衡到最极端的情况，前 1005 条数据都出在某个分表中，也可以正确获得最后的结果，但是明显的问题就是大分页，数据较多，就会导致非常大的问题，即使如 sharding-jdbc 对于合并排序的优化做得比较好，也还是需要传输那么大量的数据，并且查询也耗时，那么有没有解决方案呢，应该说有两个，或者说主要是想讲后者<br>第一个办法是像这种查询，如果业务上不需要进行跳页，而是只给下一页，那么我们就能把前一次的最大偏移量的 create_time 记录下来，下一页就可以拿着这个偏移量进行查询，这个比较简单易懂，就不多说了<br>第二个办法是看的58 沈剑的一篇文章，尝试理解讲述一下，<br>这个办法的第一步跟前面那个错误的方法或者说不准确的方法一样，先是将分页偏移量平均后在三个表里进行查询</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">t0</span><br><span class="line">334 10158 nick95 18 1641548941767</span><br><span class="line">335 10098 nick11 18 1641548941879</span><br><span class="line">336 10167 nick51 18 1641548942089</span><br><span class="line">337 10167 nick3 18 1641548942119</span><br><span class="line">338 10170 nick57 18 1641548942169</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">t1</span><br><span class="line">334 10105 nick98 18 1641548939071 最小</span><br><span class="line">335 10174 nick94 18 1641548939377</span><br><span class="line">336 10129 nick85 18 1641548939442</span><br><span class="line">337 10141 nick84 18 1641548939480</span><br><span class="line">338 10096 nick74 18 1641548939668</span><br><span class="line"></span><br><span class="line">t2</span><br><span class="line">334 10184 nick11 18 1641548945075</span><br><span class="line">335 10109 nick93 18 1641548945382</span><br><span class="line">336 10181 nick41 18 1641548945583</span><br><span class="line">337 10130 nick80 18 1641548945993</span><br><span class="line">338 10184 nick19 18 1641548946294 最大</span><br></pre></td></tr></table></figure>
|
|
<p>然后要做什么呢，其实目标比较明白，因为前面那种方法其实就是我知道了前一页的偏移量，所以可以直接当做条件来进行查询，那这里我也想着拿到这个条件，所以我将第一遍查出来的最小的 create_time 和最大的 create_time 找出来，然后再去三个表里查询，其实主要是最小值，因为我拿着最小值去查以后我就能知道这个最小值在每个表里处在什么位置，</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">t0</span><br><span class="line">322 10161 nick81 18 1641548939284</span><br><span class="line">323 10113 nick16 18 1641548939393</span><br><span class="line">324 10110 nick56 18 1641548939577</span><br><span class="line">325 10116 nick69 18 1641548939588</span><br><span class="line">326 10173 nick51 18 1641548939646</span><br><span class="line"></span><br><span class="line">t1</span><br><span class="line">334 10105 nick98 18 1641548939071</span><br><span class="line">335 10174 nick94 18 1641548939377</span><br><span class="line">336 10129 nick85 18 1641548939442</span><br><span class="line">337 10141 nick84 18 1641548939480</span><br><span class="line">338 10096 nick74 18 1641548939668</span><br><span class="line"></span><br><span class="line">t2</span><br><span class="line">297 10136 nick28 18 1641548939161</span><br><span class="line">298 10142 nick68 18 1641548939177</span><br><span class="line">299 10124 nick41 18 1641548939237</span><br><span class="line">300 10148 nick87 18 1641548939510</span><br><span class="line">301 10169 nick23 18 1641548939715</span><br></pre></td></tr></table></figure>
|
|
<p>我只贴了前五条数据，为了方便知道偏移量，每个分表都使用了自增主键，我们可以看到前一次查询的最小值分别在其他两个表里的位置分别是 322-1 和 297-1，那么对于总体来说这个时间应该是在 <code>322 - 1 + 333 + 297 - 1 = 951</code>，那这样子我只要对后面的数据最多每个表查 <code>1000 - 951 + 5 = 54</code> 条数据再进行合并排序就可以获得最终正确的结果。<br>这个就是传说中的二次查询法。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Sharding-Jdbc</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 Sharding-Jdbc 的简单使用</title>
|
|
<url>/2021/12/12/%E8%81%8A%E8%81%8A-Sharding-Jdbc-%E7%9A%84%E7%AE%80%E5%8D%95%E4%BD%BF%E7%94%A8/</url>
|
|
<content><![CDATA[<p>我们在日常工作中还是使用比较多的分库分表组件的，其中比较优秀的就有 Sharding-Jdbc，一开始由当当开源，后来捐献给了 Apache，说一下简单使用，因为原来经常的使用都是基于 xml 跟 properties 组合起来使用，这里主要试下用 Java Config 来配置<br>首先是通过 Spring Initializr 创建个带 jdbc 的 Spring Boot 项目，然后引入主要的依赖</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.apache.shardingsphere<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>shardingsphere-jdbc-core<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>5.0.0-beta<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"><span class="tag"></<span class="name">dependency</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>因为前面有聊过 Spring Boot 的自动加载，在这里 spring 就会自己去找 DataSource 的配置，所以要在入口把它干掉</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@SpringBootApplication(exclude = { DataSourceAutoConfiguration.class})</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">ShardingJdbcDemoApplication</span> <span class="keyword">implements</span> <span class="title class_">CommandLineRunner</span> {</span><br></pre></td></tr></table></figure>
|
|
<p>然后因为想在入口跑代码，就实现了下 <code>org.springframework.boot.CommandLineRunner</code> 主要是后面的 Java Config 代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"></span><br><span class="line"><span class="comment">// 注意这里的注解，可以让 Spring 自动帮忙加载，也就是 Java Config 的核心</span></span><br><span class="line"><span class="meta">@Configuration</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">MysqlConfig</span> {</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Bean</span></span><br><span class="line"> <span class="keyword">public</span> DataSource <span class="title function_">dataSource</span><span class="params">()</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="comment">// Configure actual data sources</span></span><br><span class="line"> Map<String, DataSource> dataSourceMap = <span class="keyword">new</span> <span class="title class_">HashMap</span><>();</span><br><span class="line"></span><br><span class="line"> </span><br><span class="line"> <span class="comment">// Configure the first data source</span></span><br><span class="line"> <span class="comment">// 使用了默认的Hikari连接池的 DataSource</span></span><br><span class="line"> <span class="type">HikariDataSource</span> <span class="variable">dataSource1</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">HikariDataSource</span>();</span><br><span class="line"> dataSource1.setDriverClassName(<span class="string">"com.mysql.jdbc.Driver"</span>);</span><br><span class="line"> dataSource1.setJdbcUrl(<span class="string">"jdbc:mysql://localhost:3306/sharding"</span>);</span><br><span class="line"> dataSource1.setUsername(<span class="string">"username"</span>);</span><br><span class="line"> dataSource1.setPassword(<span class="string">"password"</span>);</span><br><span class="line"> dataSourceMap.put(<span class="string">"ds0"</span>, dataSource1);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Configure student table rule</span></span><br><span class="line"> <span class="comment">// 这里是配置分表逻辑，逻辑表是 student，对应真实的表是 student_0 到 student_1, 这个配置方式就是有多少表可以用 student_$->{0..n}</span></span><br><span class="line"> <span class="type">ShardingTableRuleConfiguration</span> <span class="variable">studentTableRuleConfig</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ShardingTableRuleConfiguration</span>(<span class="string">"student"</span>, <span class="string">"ds0.student_$->{0..1}"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 设置分表字段</span></span><br><span class="line"> studentTableRuleConfig.setTableShardingStrategy(<span class="keyword">new</span> <span class="title class_">StandardShardingStrategyConfiguration</span>(<span class="string">"user_id"</span>, <span class="string">"tableShardingAlgorithm"</span>));</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">// Configure sharding rule</span></span><br><span class="line"> <span class="comment">// 配置 studentTableRuleConfig</span></span><br><span class="line"> <span class="type">ShardingRuleConfiguration</span> <span class="variable">shardingRuleConfig</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ShardingRuleConfiguration</span>();</span><br><span class="line"> shardingRuleConfig.getTables().add(studentTableRuleConfig);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Configure table sharding algorithm</span></span><br><span class="line"> <span class="type">Properties</span> <span class="variable">tableShardingAlgorithmrProps</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Properties</span>();</span><br><span class="line"> <span class="comment">// 算法表达式就是根据 user_id 对 2 进行取模</span></span><br><span class="line"> tableShardingAlgorithmrProps.setProperty(<span class="string">"algorithm-expression"</span>, <span class="string">"student_${user_id % 2}"</span>);</span><br><span class="line"> shardingRuleConfig.getShardingAlgorithms().put(<span class="string">"tableShardingAlgorithm"</span>, <span class="keyword">new</span> <span class="title class_">ShardingSphereAlgorithmConfiguration</span>(<span class="string">"INLINE"</span>, tableShardingAlgorithmrProps));</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">// 然后创建这个 DataSource</span></span><br><span class="line"> <span class="keyword">return</span> ShardingSphereDataSourceFactory.createDataSource(dataSourceMap, Collections.singleton(shardingRuleConfig), <span class="keyword">new</span> <span class="title class_">Properties</span>());</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>然后我们就可以在使用这个 DataSource 了，先看下这两个表的数据<br><img data-src="https://img.nicksxs.me/uPic/LAcX4H.png"><br><img data-src="https://img.nicksxs.me/uPic/HB4wb1.png"></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">run</span><span class="params">(String... args)</span> {</span><br><span class="line"> LOGGER.info(<span class="string">"run here"</span>);</span><br><span class="line"> <span class="type">String</span> <span class="variable">sql</span> <span class="operator">=</span> <span class="string">"SELECT * FROM student WHERE user_id=? "</span>;</span><br><span class="line"> <span class="keyword">try</span> (</span><br><span class="line"> <span class="type">Connection</span> <span class="variable">conn</span> <span class="operator">=</span> dataSource.getConnection();</span><br><span class="line"> <span class="type">PreparedStatement</span> <span class="variable">ps</span> <span class="operator">=</span> conn.prepareStatement(sql)) {</span><br><span class="line"> <span class="comment">// 参数就是 user_id，然后也是分表键，对 2 取模就是 1，应该是去 student_1 取数据</span></span><br><span class="line"> ps.setInt(<span class="number">1</span>, <span class="number">1001</span>);</span><br><span class="line"></span><br><span class="line"> <span class="type">ResultSet</span> <span class="variable">resultSet</span> <span class="operator">=</span> ps.executeQuery();</span><br><span class="line"> <span class="keyword">while</span> (resultSet.next()) {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">id</span> <span class="operator">=</span> resultSet.getInt(<span class="string">"id"</span>);</span><br><span class="line"> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">name</span> <span class="operator">=</span> resultSet.getString(<span class="string">"name"</span>);</span><br><span class="line"> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">userId</span> <span class="operator">=</span> resultSet.getInt(<span class="string">"user_id"</span>);</span><br><span class="line"> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">age</span> <span class="operator">=</span> resultSet.getInt(<span class="string">"age"</span>);</span><br><span class="line"> System.out.println(<span class="string">"奇数表 id："</span> + id + <span class="string">" 姓名："</span> + name</span><br><span class="line"> + <span class="string">" 用户 id："</span> + userId + <span class="string">" 年龄："</span> + age );</span><br><span class="line"> System.out.println(<span class="string">"============================="</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 参数就是 user_id，然后也是分表键，对 2 取模就是 0，应该是去 student_0 取数据</span></span><br><span class="line"> ps.setInt(<span class="number">1</span>, <span class="number">1000</span>);</span><br><span class="line"> resultSet = ps.executeQuery();</span><br><span class="line"> <span class="keyword">while</span> (resultSet.next()) {</span><br><span class="line"> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">id</span> <span class="operator">=</span> resultSet.getInt(<span class="string">"id"</span>);</span><br><span class="line"> <span class="keyword">final</span> <span class="type">String</span> <span class="variable">name</span> <span class="operator">=</span> resultSet.getString(<span class="string">"name"</span>);</span><br><span class="line"> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">userId</span> <span class="operator">=</span> resultSet.getInt(<span class="string">"user_id"</span>);</span><br><span class="line"> <span class="keyword">final</span> <span class="type">int</span> <span class="variable">age</span> <span class="operator">=</span> resultSet.getInt(<span class="string">"age"</span>);</span><br><span class="line"> System.out.println(<span class="string">"偶数表 id："</span> + id + <span class="string">" 姓名："</span> + name</span><br><span class="line"> + <span class="string">" 用户 id："</span> + userId + <span class="string">" 年龄："</span> + age );</span><br><span class="line"> System.out.println(<span class="string">"============================="</span>);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (SQLException e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>看下查询结果<br><img data-src="https://img.nicksxs.me/uPic/hudOBb.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Sharding-Jdbc</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 Sharding-Jdbc 的简单原理初篇</title>
|
|
<url>/2021/12/26/%E8%81%8A%E8%81%8A-Sharding-Jdbc-%E7%9A%84%E7%AE%80%E5%8D%95%E5%8E%9F%E7%90%86%E5%88%9D%E7%AF%87/</url>
|
|
<content><![CDATA[<p>在上一篇 sharding-jdbc 的介绍中其实碰到过一个问题，这里也引出了一个比较有意思的话题<br>就是我在执行 query 的时候犯过一个比较难发现的错误，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">ResultSet</span> <span class="variable">resultSet</span> <span class="operator">=</span> ps.executeQuery(sql);</span><br></pre></td></tr></table></figure>
|
|
<p>实际上应该是</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">ResultSet</span> <span class="variable">resultSet</span> <span class="operator">=</span> ps.executeQuery();</span><br></pre></td></tr></table></figure>
|
|
<p>而这里的差别就是，是否传 sql 这个参数，首先我们要知道这个 ps 是什么，它也是个接口<code>java.sql.PreparedStatement</code>,而真正的实现类是<code>org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSpherePreparedStatement</code>,我们来看下继承关系<br><img data-src="https://img.nicksxs.me/uPic/LqJznh.png"><br>这里可以看到继承关系里有<code>org.apache.shardingsphere.driver.jdbc.unsupported.AbstractUnsupportedOperationPreparedStatement</code><br>那么在我上面的写错的代码里</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">final</span> ResultSet <span class="title function_">executeQuery</span><span class="params">(<span class="keyword">final</span> String sql)</span> <span class="keyword">throws</span> SQLException {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">SQLFeatureNotSupportedException</span>(<span class="string">"executeQuery with SQL for PreparedStatement"</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这个报错一开始让我有点懵，后来点进去了发现是这么个异常，但是我其实一开始是用的更新语句，以为更新不支持，因为平时使用没有深究过，以为是不是需要使用 Mybatis 才可以执行更新，但是理论上也不应该，再往上看原来这些异常是由 sharding-jdbc 包装的，也就是在上面说的<code>AbstractUnsupportedOperationPreparedStatement</code>，这其实也是一种设计思想，本身 jdbc 提供了一系列接口，由各家去支持，包括 mysql，sql server，oracle 等，而正因为这个设计，所以 sharding-jdbc 也可以在此基础上进行设计，我们可以总体地看下 sharding-jdbc 的实现基础<br><img data-src="https://img.nicksxs.me/uPic/RWvrK6.png"><br>看了前面<code>ShardingSpherePreparedStatement</code>的继承关系，应该也能猜到这里的几个类都是实现了 jdbc 的基础接口，<br><img data-src="https://img.nicksxs.me/uPic/0QncpD.png"><br>在前一篇的 demo 中的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">Connection</span> <span class="variable">conn</span> <span class="operator">=</span> dataSource.getConnection();</span><br></pre></td></tr></table></figure>
|
|
<p>其实就获得了<code>org.apache.shardingsphere.driver.jdbc.core.connection.ShardingSphereConnection#ShardingSphereConnection</code><br>然后获得<code>java.sql.PreparedStatement</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">PreparedStatement</span> <span class="variable">ps</span> <span class="operator">=</span> conn.prepareStatement(sql)</span><br></pre></td></tr></table></figure>
|
|
<p>就是获取了<code>org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSpherePreparedStatement</code><br>然后就是执行</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">ResultSet</span> <span class="variable">resultSet</span> <span class="operator">=</span> ps.executeQuery();</span><br></pre></td></tr></table></figure>
|
|
<p>然后获得结果<br><code>org.apache.shardingsphere.driver.jdbc.core.resultset.ShardingSphereResultSet</code></p>
|
|
<p>其实像 mybatis 也是基于这样去实现的</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Sharding-Jdbc</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 dubbo 的线程池</title>
|
|
<url>/2021/04/04/%E8%81%8A%E8%81%8A-dubbo-%E7%9A%84%E7%BA%BF%E7%A8%8B%E6%B1%A0/</url>
|
|
<content><![CDATA[<p>之前没注意到这一块，只是比较模糊的印象 dubbo 自己基于 ThreadPoolExecutor 定义了几个线程池，但是没具体看过，主要是觉得就是为了避免使用 jdk 自带的那几个(java.util.concurrent.Executors)，防止出现那些问题<br>看下代码目录主要是这几个<br><img data-src="https://img.nicksxs.me/uPic/3qIllK.png"></p>
|
|
<ul>
|
|
<li>FixedThreadPool：创建一个复用固定个数线程的线程池。<br>简单看下代码<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Executor <span class="title function_">getExecutor</span><span class="params">(URL url)</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">name</span> <span class="operator">=</span> url.getParameter(<span class="string">"threadname"</span>, <span class="string">"Dubbo"</span>);</span><br><span class="line"> <span class="type">int</span> <span class="variable">threads</span> <span class="operator">=</span> url.getParameter(<span class="string">"threads"</span>, <span class="number">200</span>);</span><br><span class="line"> <span class="type">int</span> <span class="variable">queues</span> <span class="operator">=</span> url.getParameter(<span class="string">"queues"</span>, <span class="number">0</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">ThreadPoolExecutor</span>(threads, threads, <span class="number">0L</span>, TimeUnit.MILLISECONDS, (BlockingQueue)(queues == <span class="number">0</span> ? <span class="keyword">new</span> <span class="title class_">SynchronousQueue</span>() : (queues < <span class="number">0</span> ? <span class="keyword">new</span> <span class="title class_">LinkedBlockingQueue</span>() : <span class="keyword">new</span> <span class="title class_">LinkedBlockingQueue</span>(queues))), <span class="keyword">new</span> <span class="title class_">NamedThreadFactory</span>(name, <span class="literal">true</span>), <span class="keyword">new</span> <span class="title class_">AbortPolicyWithReport</span>(name, url));</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
可以看到核心线程数跟最大线程数一致，也就是说就不会在核心线程数和最大线程数之间动态变化了</li>
|
|
<li>LimitedThreadPool：创建一个线程池，这个线程池中线程个数随着需要量动态增加，但是数量不超过配置的阈值的个数，另外空闲线程不会被回收，会一直存在。<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Executor <span class="title function_">getExecutor</span><span class="params">(URL url)</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">name</span> <span class="operator">=</span> url.getParameter(<span class="string">"threadname"</span>, <span class="string">"Dubbo"</span>);</span><br><span class="line"> <span class="type">int</span> <span class="variable">cores</span> <span class="operator">=</span> url.getParameter(<span class="string">"corethreads"</span>, <span class="number">0</span>);</span><br><span class="line"> <span class="type">int</span> <span class="variable">threads</span> <span class="operator">=</span> url.getParameter(<span class="string">"threads"</span>, <span class="number">200</span>);</span><br><span class="line"> <span class="type">int</span> <span class="variable">queues</span> <span class="operator">=</span> url.getParameter(<span class="string">"queues"</span>, <span class="number">0</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">ThreadPoolExecutor</span>(cores, threads, <span class="number">9223372036854775807L</span>, TimeUnit.MILLISECONDS, (BlockingQueue)(queues == <span class="number">0</span> ? <span class="keyword">new</span> <span class="title class_">SynchronousQueue</span>() : (queues < <span class="number">0</span> ? <span class="keyword">new</span> <span class="title class_">LinkedBlockingQueue</span>() : <span class="keyword">new</span> <span class="title class_">LinkedBlockingQueue</span>(queues))), <span class="keyword">new</span> <span class="title class_">NamedThreadFactory</span>(name, <span class="literal">true</span>), <span class="keyword">new</span> <span class="title class_">AbortPolicyWithReport</span>(name, url));</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
这个特点主要是创建了保活时间特别长，即可以认为不会被回收了</li>
|
|
<li>EagerThreadPool ：创建一个线程池，这个线程池当所有核心线程都处于忙碌状态时候，创建新的线程来执行新任务，而不是把任务放入线程池阻塞队列。<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Executor <span class="title function_">getExecutor</span><span class="params">(URL url)</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">name</span> <span class="operator">=</span> url.getParameter(<span class="string">"threadname"</span>, <span class="string">"Dubbo"</span>);</span><br><span class="line"> <span class="type">int</span> <span class="variable">cores</span> <span class="operator">=</span> url.getParameter(<span class="string">"corethreads"</span>, <span class="number">0</span>);</span><br><span class="line"> <span class="type">int</span> <span class="variable">threads</span> <span class="operator">=</span> url.getParameter(<span class="string">"threads"</span>, <span class="number">2147483647</span>);</span><br><span class="line"> <span class="type">int</span> <span class="variable">queues</span> <span class="operator">=</span> url.getParameter(<span class="string">"queues"</span>, <span class="number">0</span>);</span><br><span class="line"> <span class="type">int</span> <span class="variable">alive</span> <span class="operator">=</span> url.getParameter(<span class="string">"alive"</span>, <span class="number">60000</span>);</span><br><span class="line"> TaskQueue<Runnable> taskQueue = <span class="keyword">new</span> <span class="title class_">TaskQueue</span>(queues <= <span class="number">0</span> ? <span class="number">1</span> : queues);</span><br><span class="line"> <span class="type">EagerThreadPoolExecutor</span> <span class="variable">executor</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">EagerThreadPoolExecutor</span>(cores, threads, (<span class="type">long</span>)alive, TimeUnit.MILLISECONDS, taskQueue, <span class="keyword">new</span> <span class="title class_">NamedThreadFactory</span>(name, <span class="literal">true</span>), <span class="keyword">new</span> <span class="title class_">AbortPolicyWithReport</span>(name, url));</span><br><span class="line"> taskQueue.setExecutor(executor);</span><br><span class="line"> <span class="keyword">return</span> executor;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
这个是改动最多的一个了，因为需要实现这个机制，有兴趣的可以详细看下</li>
|
|
<li>CachedThreadPool： 创建一个自适应线程池，当线程处于空闲1分钟时候，线程会被回收，当有新请求到来时候会创建新线程<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Executor <span class="title function_">getExecutor</span><span class="params">(URL url)</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">name</span> <span class="operator">=</span> url.getParameter(<span class="string">"threadname"</span>, <span class="string">"Dubbo"</span>);</span><br><span class="line"> <span class="type">int</span> <span class="variable">cores</span> <span class="operator">=</span> url.getParameter(<span class="string">"corethreads"</span>, <span class="number">0</span>);</span><br><span class="line"> <span class="type">int</span> <span class="variable">threads</span> <span class="operator">=</span> url.getParameter(<span class="string">"threads"</span>, <span class="number">2147483647</span>);</span><br><span class="line"> <span class="type">int</span> <span class="variable">queues</span> <span class="operator">=</span> url.getParameter(<span class="string">"queues"</span>, <span class="number">0</span>);</span><br><span class="line"> <span class="type">int</span> <span class="variable">alive</span> <span class="operator">=</span> url.getParameter(<span class="string">"alive"</span>, <span class="number">60000</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">ThreadPoolExecutor</span>(cores, threads, (<span class="type">long</span>)alive, TimeUnit.MILLISECONDS, (BlockingQueue)(queues == <span class="number">0</span> ? <span class="keyword">new</span> <span class="title class_">SynchronousQueue</span>() : (queues < <span class="number">0</span> ? <span class="keyword">new</span> <span class="title class_">LinkedBlockingQueue</span>() : <span class="keyword">new</span> <span class="title class_">LinkedBlockingQueue</span>(queues))), <span class="keyword">new</span> <span class="title class_">NamedThreadFactory</span>(name, <span class="literal">true</span>), <span class="keyword">new</span> <span class="title class_">AbortPolicyWithReport</span>(name, url));</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
这里可以看到线程池的配置，核心是 0，最大线程数是 2147483647，保活时间是一分钟<br>只是非常简略的介绍下，有兴趣可以自行阅读代码。</li>
|
|
</ul>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Dubbo - 线程池</category>
|
|
<category>Dubbo</category>
|
|
<category>线程池</category>
|
|
<category>ThreadPool</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Dubbo</tag>
|
|
<tag>ThreadPool</tag>
|
|
<tag>线程池</tag>
|
|
<tag>FixedThreadPool</tag>
|
|
<tag>LimitedThreadPool</tag>
|
|
<tag>EagerThreadPool</tag>
|
|
<tag>CachedThreadPool</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 mysql 的 MVCC 续续篇之锁分析</title>
|
|
<url>/2020/05/10/%E8%81%8A%E8%81%8A-mysql-%E7%9A%84-MVCC-%E7%BB%AD%E7%BB%AD%E7%AF%87%E4%B9%8B%E5%8A%A0%E9%94%81%E5%88%86%E6%9E%90/</url>
|
|
<content><![CDATA[<p>看完前面两篇水文之后，感觉不得不来分析下 mysql 的锁了，其实前面说到幻读的时候是有个前提没提到的，比如一个<code>select * from table1 where id = 1</code>这种查询语句其实是不会加传说中的锁的，当然这里是指在 RR 或者 RC 隔离级别下，<br>看一段 mysql官方文档</p>
|
|
<blockquote>
|
|
<p><code>SELECT ... FROM</code> is a consistent read, reading a snapshot of the database and setting no locks unless the transaction isolation level is set to SERIALIZABLE. For SERIALIZABLE level, the search sets shared next-key locks on the index records it encounters. However, only an index record lock is required for statements that lock rows using a unique index to search for a unique row. </p>
|
|
</blockquote>
|
|
<p>纯粹的这种一致性读，实际读取的是快照，也就是基于 read view 的读取方式，除非当前隔离级别是SERIALIZABLE<br>但是对于以下几类</p>
|
|
<ul>
|
|
<li><code>select * from table where ? lock in share mode;</code></li>
|
|
<li><code>select * from table where ? for update;</code></li>
|
|
<li><code>insert into table values (...);</code></li>
|
|
<li><code>update table set ? where ?;</code></li>
|
|
<li><code>delete from table where ?;</code></li>
|
|
</ul>
|
|
<p>除了第一条是 S 锁之外，其他都是 X 排他锁，这边在顺带下，S 锁表示共享锁， X 表示独占锁，同为 S 锁之间不冲突，S 与 X，X 与 S，X 与 X 之间都冲突，也就是加了前者，后者就加不上了<br>我们知道对于 RC 级别会出现幻读现象，对于 RR 级别不会出现，主要的区别是 RR 级别下对于以上的加锁读取都根据情况加上了 gap 锁，那么是不是 RR 级别下以上所有的都是要加 gap 锁呢，当然不是<br>举个例子，RR 事务隔离级别下，table1 有个主键id 字段<br><code>select * from table1 where id = 10 for update</code><br>这条语句要加 gap 锁吗？<br>答案是不需要，这里其实算是我看了这么久的一点自己的理解，啥时候要加 gap 锁，判断的条件是根据我查询的数据是否会因为不加 gap 锁而出现数量的不一致，我上面这条查询语句，在什么情况下会出现查询结果数量不一致呢，只要在这条记录被更新或者删除的时候，有没有可能我第一次查出来一条，第二次变成两条了呢，不可能，因为是主键索引。<br>再变更下这个题的条件，当 id 不是主键，但是是唯一索引，这样需要怎么加锁，注意问题是怎么加锁，不是需不需要加 gap 锁，这里呢就是稍微延伸一下，把聚簇索引（主键索引）和二级索引带一下，当 id 不是主键，说明是个二级索引，但是它是唯一索引，体会下，首先对于 id = 10这个二级索引肯定要加锁，要不要锁 gap 呢，不用，因为是唯一索引，id = 10 只可能有这一条记录，然后呢，这样是不是就好了，还不行，因为啥，因为它是二级索引，对应的主键索引的记录才是真正的数据，万一被更新掉了咋办，所以在 id = 10 对应的主键索引上也需要加上锁（默认都是 record lock行锁），那主键索引上要不要加 gap 呢，也不用，也是精确定位到这一条记录<br>最后呢，当 id 不是主键，也不是唯一索引，只是个普通的索引，这里就需要大名鼎鼎的 gap 锁了，<br>是时候画个图了<br><img data-src="https://mystore-1255223546.cos.ap-chengdu.myqcloud.com/uPic/WX20200510-1126082x.png"><br>其实核心的目的还是不让这个 id=10 的记录不会出现幻读，那么就需要在 id 这个索引上加上三个 gap 锁，主键索引上就不用了，在 id 索引上已经控制住了id = 10 不会出现幻读，主键索引上这两条对应的记录已经锁了，所以就这样 OK 了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Mysql</category>
|
|
<category>C</category>
|
|
<category>数据结构</category>
|
|
<category>源码</category>
|
|
<category>Mysql</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>mysql</tag>
|
|
<tag>数据结构</tag>
|
|
<tag>源码</tag>
|
|
<tag>mvcc</tag>
|
|
<tag>read view</tag>
|
|
<tag>gap lock</tag>
|
|
<tag>next-key lock</tag>
|
|
<tag>幻读</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 mysql 的 MVCC 续篇</title>
|
|
<url>/2020/05/02/%E8%81%8A%E8%81%8A-mysql-%E7%9A%84-MVCC-%E7%BB%AD%E7%AF%87/</url>
|
|
<content><![CDATA[<p>上一篇聊了mysql 的 innodb 引擎基于 read view 实现的 mvcc 和事务隔离级别，可能有些细心的小伙伴会发现一些问题，第一个是在 RC 级别下的事务提交后的可见性，这里涉及到了三个参数，m_low_limit_id，m_up_limit_id，m_ids，之前看到知乎的一篇写的非常不错的文章，但是就在这一点上似乎有点疑惑，这里基于源码和注释来解释下这个问题</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment">Opens a read view where exactly the transactions serialized before this</span></span><br><span class="line"><span class="comment">point in time are seen in the view.</span></span><br><span class="line"><span class="comment">@param id Creator transaction id */</span></span><br><span class="line"></span><br><span class="line"><span class="type">void</span> <span class="title function_">ReadView::prepare</span><span class="params">(<span class="type">trx_id_t</span> id)</span> {</span><br><span class="line"> ut_ad(mutex_own(&trx_sys->mutex));</span><br><span class="line"></span><br><span class="line"> m_creator_trx_id = id;</span><br><span class="line"></span><br><span class="line"> m_low_limit_no = m_low_limit_id = m_up_limit_id = trx_sys->max_trx_id;</span><br></pre></td></tr></table></figure>
|
|
<p>m_low_limit_id赋的值是trx_sys->max_trx_id，代表的是当前系统最小的未分配的事务 id，所以呢，举个例子，当前有三个活跃事务，事务 id 分别是 100，200，300，而 m_up_limit_id = 100, m_low_limit_id = 301，当事务 id 是 200 的提交之后，它的更新就是可以被 100 和 300 看到，而不是说 m_ids 里没了 200，并且 200 比 100 大就应该不可见了</p>
|
|
<h2 id="幻读"><a href="#幻读" class="headerlink" title="幻读"></a>幻读</h2><p>还有一个问题是幻读的问题，这貌似也是个高频面试题，啥意思呢，或者说跟它最常拿来比较的脏读，脏读是指读到了别的事务未提交的数据，因为未提交，严格意义上来讲，不一定是会被最后落到库里，可能会回滚，也就是在 read uncommitted 级别下会出现的问题，但是幻读不太一样，幻读是指两次查询的结果数量不一样，比如我查了第一次是 <code>select * from table1 where id < 10 for update</code>,查出来了一条结果 id 是 5，然后再查一下发现出来了一条 id 是 5，一条 id 是 7，那是不是有点尴尬了，其实呢这个点我觉得脏读跟幻读也比较是从原理层面来命名，如果第一次接触的同学发觉有点不理解也比较正常，因为从逻辑上讲总之都是数据不符合预期，但是基于源码层面其实是不同的情况，幻读是在原先的 read view 无法完全解决的，怎么解决呢，简单的来说就是锁咯，我们知道innodb 是基于 record lock 行锁的，但是貌似没有办法解决这种问题，那么 innodb 就引入了 gap lock 间隙锁，比如上面说的情况下，id 小于 10 的情况下，是都应该锁住的，gap lock 其实是基于索引结构来锁的，因为索引树除了树形结构之外，还有一个next record 的指针，gap lock 也是基于这个来锁的<br>看一下 mysql 的文档</p>
|
|
<blockquote>
|
|
<p>SELECT … FOR UPDATE sets an exclusive next-key lock on every record the search encounters. However, only an index record lock is required for statements that lock rows using a unique index to search for a unique row. </p>
|
|
</blockquote>
|
|
<p>对于一个 for update 查询，在 RR 级别下，会设置一个 next-key lock在每一条被查询到的记录上，next-lock 又是啥呢，其实就是 gap 锁和 record 锁的结合体，比如我在数据库里有 id 是 1，3，5，7，10，对于上面那条查询，查出来的结果就是 1，3，5，7，那么按照文档里描述的，对于这几条记录都会加上next-key lock，也就是(-∞, 1], (1, 3], (3, 5], (5, 7], (7, 10) 这些区间和记录会被锁起来，不让插入，再唠叨一下呢，就是其实如果是只读的事务，光 read view 一致性读就够了，如果是有写操作的呢，就需要锁了。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Mysql</category>
|
|
<category>C</category>
|
|
<category>数据结构</category>
|
|
<category>源码</category>
|
|
<category>Mysql</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>mysql</tag>
|
|
<tag>数据结构</tag>
|
|
<tag>源码</tag>
|
|
<tag>mvcc</tag>
|
|
<tag>read view</tag>
|
|
<tag>gap lock</tag>
|
|
<tag>next-key lock</tag>
|
|
<tag>幻读</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 mysql 索引的一些细节</title>
|
|
<url>/2020/12/27/%E8%81%8A%E8%81%8A-mysql-%E7%B4%A2%E5%BC%95%E7%9A%84%E4%B8%80%E4%BA%9B%E7%BB%86%E8%8A%82/</url>
|
|
<content><![CDATA[<p>前几天同事问了我个 mysql 索引的问题，虽然大概知道，但是还是想来实践下，就是 is null，is not null 这类查询是否能用索引，可能之前有些网上的文章说都是不能用索引，但是其实不是，我们来看个小试验</p>
|
|
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> `null_index_t` (</span><br><span class="line"> `id` <span class="type">int</span>(<span class="number">10</span>) unsigned <span class="keyword">NOT</span> <span class="keyword">NULL</span> AUTO_INCREMENT,</span><br><span class="line"> `null_key` <span class="type">varchar</span>(<span class="number">255</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `null_key1` <span class="type">varchar</span>(<span class="number">255</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `null_key2` <span class="type">varchar</span>(<span class="number">255</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> <span class="keyword">PRIMARY</span> KEY (`id`),</span><br><span class="line"> KEY `idx_1` (`null_key`) <span class="keyword">USING</span> BTREE,</span><br><span class="line"> KEY `idx_2` (`null_key1`) <span class="keyword">USING</span> BTREE,</span><br><span class="line"> KEY `idx_3` (`null_key2`) <span class="keyword">USING</span> BTREE</span><br><span class="line">) ENGINE<span class="operator">=</span>InnoDB <span class="keyword">DEFAULT</span> CHARSET<span class="operator">=</span>utf8mb4;</span><br></pre></td></tr></table></figure>
|
|
<p>用个存储过程来插入数据</p>
|
|
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"></span><br><span class="line">delimiter $ #以delimiter来标记用$表示存储过程结束</span><br><span class="line"><span class="keyword">create</span> <span class="keyword">procedure</span> nullIndex1()</span><br><span class="line"><span class="keyword">begin</span></span><br><span class="line"><span class="keyword">declare</span> i <span class="type">int</span>; </span><br><span class="line"><span class="keyword">declare</span> j <span class="type">int</span>; </span><br><span class="line"><span class="keyword">set</span> i<span class="operator">=</span><span class="number">1</span>;</span><br><span class="line"><span class="keyword">set</span> j<span class="operator">=</span><span class="number">1</span>;</span><br><span class="line">while(i<span class="operator"><=</span><span class="number">100</span>) do </span><br><span class="line"> while(j<span class="operator"><=</span><span class="number">100</span>) do </span><br><span class="line"> IF (i <span class="operator">%</span> <span class="number">3</span> <span class="operator">=</span> <span class="number">0</span>) <span class="keyword">THEN</span></span><br><span class="line"> <span class="keyword">INSERT</span> <span class="keyword">INTO</span> null_index_t ( `null_key`, `null_key1`, `null_key2` ) <span class="keyword">VALUES</span> (<span class="keyword">null</span> , <span class="keyword">LEFT</span>(MD5(RAND()), <span class="number">8</span>), <span class="keyword">LEFT</span>(MD5(RAND()), <span class="number">8</span>));</span><br><span class="line"> ELSEIF (i <span class="operator">%</span> <span class="number">3</span> <span class="operator">=</span> <span class="number">1</span>) <span class="keyword">THEN</span></span><br><span class="line"> <span class="keyword">INSERT</span> <span class="keyword">INTO</span> null_index_t ( `null_key`, `null_key1`, `null_key2` ) <span class="keyword">VALUES</span> (<span class="keyword">LEFT</span>(MD5(RAND()), <span class="number">8</span>), <span class="keyword">NULL</span>, <span class="keyword">LEFT</span>(MD5(RAND()), <span class="number">8</span>));</span><br><span class="line"> <span class="keyword">ELSE</span></span><br><span class="line"> <span class="keyword">INSERT</span> <span class="keyword">INTO</span> null_index_t ( `null_key`, `null_key1`, `null_key2` ) <span class="keyword">VALUES</span> (<span class="keyword">LEFT</span>(MD5(RAND()), <span class="number">8</span>), <span class="keyword">LEFT</span>(MD5(RAND()), <span class="number">8</span>), <span class="keyword">NULL</span>);</span><br><span class="line"> <span class="keyword">END</span> IF;</span><br><span class="line"> <span class="keyword">set</span> j<span class="operator">=</span>j<span class="operator">+</span><span class="number">1</span>;</span><br><span class="line"> <span class="keyword">end</span> while;</span><br><span class="line"> <span class="keyword">set</span> i<span class="operator">=</span>i<span class="operator">+</span><span class="number">1</span>;</span><br><span class="line"> <span class="keyword">set</span> j<span class="operator">=</span><span class="number">1</span>; </span><br><span class="line"><span class="keyword">end</span> while;</span><br><span class="line"><span class="keyword">end</span> </span><br><span class="line">$</span><br><span class="line"><span class="keyword">call</span> nullIndex1();</span><br></pre></td></tr></table></figure>
|
|
<p>然后看下我们的 is null 查询</p>
|
|
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line">EXPLAIN <span class="keyword">select</span> <span class="operator">*</span> <span class="keyword">from</span> null_index_t <span class="keyword">WHERE</span> null_key <span class="keyword">is</span> <span class="keyword">null</span>;</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/uPic/IejArR.png"><br>再来看看另一个</p>
|
|
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line">EXPLAIN <span class="keyword">select</span> <span class="operator">*</span> <span class="keyword">from</span> null_index_t <span class="keyword">WHERE</span> null_key <span class="keyword">is</span> <span class="keyword">not</span> <span class="keyword">null</span>;</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/uPic/vwttcE.png"><br>从这里能看出来啥呢，可以思考下</p>
|
|
<p>从上面可以发现，<code>is null</code>应该是用上了索引了，所以至少不是一刀切不能用，但是看着<code>is not null</code>好像不太行额<br>我们在做一点小改动，把这个表里的数据改成 9100 条是 null，剩下 900 条是有值的，然后再执行下<br><img data-src="https://img.nicksxs.me/uPic/McIoej.png"><br>然后再来看看执行结果</p>
|
|
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line">EXPLAIN <span class="keyword">select</span> <span class="operator">*</span> <span class="keyword">from</span> null_index_t <span class="keyword">WHERE</span> null_key <span class="keyword">is</span> <span class="keyword">null</span>;</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/uPic/i4ki84.png"></p>
|
|
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line">EXPLAIN <span class="keyword">select</span> <span class="operator">*</span> <span class="keyword">from</span> null_index_t <span class="keyword">WHERE</span> null_key <span class="keyword">is</span> <span class="keyword">not</span> <span class="keyword">null</span>;</span><br></pre></td></tr></table></figure>
|
|
<p><img data-src="https://img.nicksxs.me/uPic/1HKVQH.png"><br>是不是不一样了，这里再补充下我试验使用的 mysql 是 5.7 的，不保证在其他版本的一致性，<br>其实可以看出随着数据量的变化，mysql 会不会使用索引是会变化的，不是说 is not null 一定会使用，也不是一定不会使用，而是优化器会根据查询成本做个预判，这个预判尽可能会减小查询成本，主要包括回表啥的，但是也不一定完全准确。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Mysql</category>
|
|
<category>C</category>
|
|
<category>索引</category>
|
|
<category>Mysql</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>mysql</tag>
|
|
<tag>索引</tag>
|
|
<tag>is null</tag>
|
|
<tag>is not null</tag>
|
|
<tag>procedure</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 mysql 的 MVCC</title>
|
|
<url>/2020/04/26/%E8%81%8A%E8%81%8A-mysql-%E7%9A%84-MVCC/</url>
|
|
<content><![CDATA[<p>很久以前，有位面试官问到，你知道 mysql 的事务隔离级别吗，“额 O__O …，不太清楚”，完了之后我就去网上找相关的文章，找到了这篇<a href="https://www.cnblogs.com/zhoujinyi/p/3437475.html">MySQL 四种事务隔离级的说明</a>, 文章写得特别好，看了这个就懂了各个事务隔离级别都是啥，不过看了这个之后多思考一下的话还是会发现问题，这么神奇的事务隔离级别是怎么实现的呢</p>
|
|
<p>其中 innodb 的事务隔离用到了标题里说到的 mvcc，<a href="https://en.wikipedia.org/wiki/Multiversion_concurrency_control"><strong>Multiversion concurrency control</strong></a>, 直译过来就是多版本并发控制，先不讲这个究竟是个啥，考虑下如果纯猜测，这个事务隔离级别应该会是怎么样实现呢，愚钝的我想了下，可以在事务开始的时候拷贝一个表，这个可以支持 RR 级别，RC 级别就不支持了，而且要是个非常大的表，想想就不可行</p>
|
|
<p>腆着脸说虽然这个不可行，但是思路是对的，具体实行起来需要做一系列（肥肠多）的改动，首先根据我的理解，其实这个拷贝一个表是变成拷贝一条记录，但是如果有多个事务，那就得拷贝多次，这个问题其实可以借助版本管理系统来解释，在用版本管理系统，git 之类的之前，很原始的可能是开发完一个功能后，就打个压缩包用时间等信息命名，然后如果后面要找回这个就直接用这个压缩包的就行了，后来有了 svn，git 中心式和分布式的版本管理系统，它的一个特点是粒度可以控制到文件和代码行级别，对应的我们的 mysql 事务是不是也可以从一开始预想的表级别细化到行的级别，可能之前很多人都了解过，数据库的一行记录除了我们用户自定义的字段，还有一些额外的字段，去源码<a href="https://github.com/mysql/mysql-server/blob/8.0/storage/innobase/include/data0type.h#L170">data0type.h</a>里捞一下</p>
|
|
<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="comment">/* Precise data types for system columns and the length of those columns;</span></span><br><span class="line"><span class="comment"><span class="doctag">NOTE:</span> the values must run from 0 up in the order given! All codes must</span></span><br><span class="line"><span class="comment">be less than 256 */</span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> DATA_ROW_ID 0 <span class="comment">/* row id: a 48-bit integer */</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> DATA_ROW_ID_LEN 6 <span class="comment">/* stored length for row id */</span></span></span><br><span class="line"></span><br><span class="line"><span class="comment">/** Transaction id: 6 bytes */</span></span><br><span class="line">constexpr <span class="type">size_t</span> DATA_TRX_ID = <span class="number">1</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">/** Transaction ID type size in bytes. */</span></span><br><span class="line">constexpr <span class="type">size_t</span> DATA_TRX_ID_LEN = <span class="number">6</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">/** Rollback data pointer: 7 bytes */</span></span><br><span class="line">constexpr <span class="type">size_t</span> DATA_ROLL_PTR = <span class="number">2</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">/** Rollback data pointer type size in bytes. */</span></span><br><span class="line">constexpr <span class="type">size_t</span> DATA_ROLL_PTR_LEN = <span class="number">7</span>;</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>一个是 <code>DATA_ROW_ID</code>，这个是在数据没指定主键的时候会生成一个隐藏的，如果用户有指定主键就是主键了</p>
|
|
<p>一个是 <code>DATA_TRX_ID</code>，这个表示这条记录的事务 ID</p>
|
|
<p>还有一个是 <code>DATA_ROLL_PTR</code> 指向回滚段的指针</p>
|
|
<p>指向的回滚段其实就是我们常说的 undo log，这里面的具体结构就是个链表，在 mvcc 里会使用到这个，还有就是这个 <code>DATA_TRX_ID</code>，每条记录都记录了这个事务 ID，表示的是这条记录的当前值是被哪个事务修改的，下面就扯回事务了，我们知道 <code>Read Uncommitted</code>， 其实用不到隔离，直接读取当前值就好了，到了 <code>Read Committed</code> 级别，我们要让事务读取到提交过的值，mysql 使用了一个叫 <code>read view</code> 的玩意，它里面有这些值是我们需要注意的，</p>
|
|
<p><code>m_low_limit_id</code>, 这个是 read view 创建时最大的活跃事务 id</p>
|
|
<p><code>m_up_limit_id</code>, 这个是 read view 创建时最小的活跃事务 id</p>
|
|
<p><code>m_ids</code>, 这个是 read view 创建时所有的活跃事务 id 数组</p>
|
|
<p><code>m_creator_trx_id 这个是当前记录的创建事务 id</code></p>
|
|
<p>判断事务的可见性主要的逻辑是这样，</p>
|
|
<ol>
|
|
<li>当记录的事务 <code>id</code> 小于最小活跃事务 id，说明是可见的，</li>
|
|
<li>如果记录的事务 <code>id</code> 等于当前事务 id，说明是自己的更改，可见</li>
|
|
<li>如果记录的事务 <code>id</code> 大于最大的活跃事务 <code>id</code>, 不可见</li>
|
|
<li>如果记录的事务 <code>id</code> 介于 <code>m_low_limit_id</code> 和 <code>m_up_limit_id</code> 之间，则要判断它是否在 <code>m_ids</code> 中，如果在，不可见，如果不在，表示已提交，可见<br>具体的<a href="https://github.com/mysql/mysql-server/blob/8.0/storage/innobase/include/read0types.h#L160">代码</a>捞一下看看<figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="comment">/** Check whether the changes by id are visible.</span></span><br><span class="line"><span class="comment"> @param[in] id transaction id to check against the view</span></span><br><span class="line"><span class="comment"> @param[in] name table name</span></span><br><span class="line"><span class="comment"> @return whether the view sees the modifications of id. */</span></span><br><span class="line"> <span class="type">bool</span> <span class="title function_">changes_visible</span><span class="params">(<span class="type">trx_id_t</span> id, <span class="type">const</span> <span class="type">table_name_t</span> &name)</span> <span class="type">const</span></span><br><span class="line"> <span class="title function_">MY_ATTRIBUTE</span><span class="params">((warn_unused_result))</span> {</span><br><span class="line"> ut_ad(id > <span class="number">0</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (id < m_up_limit_id || id == m_creator_trx_id) {</span><br><span class="line"> <span class="keyword">return</span> (<span class="literal">true</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> check_trx_id_sanity(id, name);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (id >= m_low_limit_id) {</span><br><span class="line"> <span class="keyword">return</span> (<span class="literal">false</span>);</span><br><span class="line"></span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (m_ids.empty()) {</span><br><span class="line"> <span class="keyword">return</span> (<span class="literal">true</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">const</span> <span class="type">ids_t</span>::value_type *p = m_ids.data();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> (!<span class="built_in">std</span>::binary_search(p, p + m_ids.size(), id));</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
剩下来一点是啥呢，就是 <code>Read Committed</code> 和 <code>Repeated Read</code> 也不一样，那前面说的 <code>read view</code> 都能支持吗，又是怎么支持呢，假如这个 <code>read view</code> 是在事务一开始就创建，那好像能支持的只是 RR 事务隔离级别，其实呢，这是通过创建 <code>read view</code>的时机，对于 RR 级别，就是在事务的第一个 <code>select</code> 语句是创建，对于 RC 级别，是在每个 <code>select</code> 语句执行前都是创建一次，那样就可以保证能读到所有已提交的数据</li>
|
|
</ol>
|
|
]]></content>
|
|
<categories>
|
|
<category>Mysql</category>
|
|
<category>C</category>
|
|
<category>数据结构</category>
|
|
<category>源码</category>
|
|
<category>Mysql</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>mysql</tag>
|
|
<tag>数据结构</tag>
|
|
<tag>源码</tag>
|
|
<tag>mvcc</tag>
|
|
<tag>read view</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 redis 缓存的应用问题</title>
|
|
<url>/2021/01/31/%E8%81%8A%E8%81%8A-redis-%E7%BC%93%E5%AD%98%E7%9A%84%E5%BA%94%E7%94%A8%E9%97%AE%E9%A2%98/</url>
|
|
<content><![CDATA[<p>前面写过一系列的 redis 源码分析的，但是实际上很多的问题还是需要结合实际的使用，然后其实就避不开缓存使用的三个著名问题，穿透，击穿和雪崩，这三个概念也是有着千丝万缕的关系，</p>
|
|
<h3 id="缓存穿透"><a href="#缓存穿透" class="headerlink" title="缓存穿透"></a>缓存穿透</h3><p>缓存穿透是指当数据库中本身就不存在这个数据的时候，使用一般的缓存策略时访问不到缓存后就访问数据库，但是因为数据库也没数据，所以如果不做任何策略优化的话，这类数据就每次都会访问一次数据库，对数据库压力也会比较大。</p>
|
|
<h3 id="缓存击穿"><a href="#缓存击穿" class="headerlink" title="缓存击穿"></a>缓存击穿</h3><p>缓存击穿跟穿透比较类似的，都是访问缓存不在，然后去访问数据库，与穿透不一样的是击穿是在数据库中存在数据，但是可能由于第一次访问，或者缓存过期了，需要访问到数据库，这对于访问量小的情况其实算是个正常情况，但是随着请求量变高就会引发一些性能隐患。</p>
|
|
<h3 id="缓存雪崩"><a href="#缓存雪崩" class="headerlink" title="缓存雪崩"></a>缓存雪崩</h3><p>缓存雪崩就是击穿的大规模集群效应，当大量的缓存过期失效的时候，这些请求都是直接访问到数据库了，会对数据库造成很大的压力。</p>
|
|
<p>对于以上三种场景也有一些比较常见的解决方案，但也不能说是万无一失的，需要随着业务去寻找合适的方案</p>
|
|
<h3 id="解决缓存穿透"><a href="#解决缓存穿透" class="headerlink" title="解决缓存穿透"></a>解决缓存穿透</h3><p>对于数据库中就没这个数据的时候，一种是可以对这个 key 设置下空值，即以一个特定的表示是数据库不存在的，这种情况需要合理地调整过期时间，当这个 key 在数据库中有数据了的话，也需要有策略去更新这个值，并且如果这类 key 非常多，这个方法就会不太合适，就可以使用第二种方法，就是布隆过滤器，bloom filter，前置一个布隆过滤器，当这个 key 在数据库不存在的话，先用布隆过滤器挡一道，如果不在的话就直接返回了，当然布隆过滤器不是绝对的准确的</p>
|
|
<h3 id="解决缓存击穿"><a href="#解决缓存击穿" class="headerlink" title="解决缓存击穿"></a>解决缓存击穿</h3><p>当一个 key 的缓存过期了，如果大量请求过来访问这个 key，请求都会落在数据库里，这个时候就可以使用一些类似于互斥锁的方式去让一个线程去访问数据库，更新缓存，但是这里其实也有个问题，就是如果是热点 key 其实这种方式也比较危险，万一更新失败，或者更新操作的时候耗时比较久，就会有一大堆请求卡在那，这种情况可能需要有一些异步提前刷新缓存，可以结合具体场景选择方式</p>
|
|
<h3 id="解决缓存雪崩"><a href="#解决缓存雪崩" class="headerlink" title="解决缓存雪崩"></a>解决缓存雪崩</h3><p>雪崩的情况是指大批量的 key 都一起过期了，击穿的放大版，大批量的请求都打到数据库上了，一方面有可能直接缓存不可用了，就需要用集群化高可用的缓存服务，然后对于实际使用中也可以使用本地缓存结合 redis 缓存，去提高可用性，再配合一些限流措施，然后就是缓存使用过程总的过期时间最好能加一些随机值，防止在同一时间过期而导致雪崩，结合互斥锁防止大量请求打到数据库。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Redis</category>
|
|
<category>应用</category>
|
|
<category>缓存</category>
|
|
<category>缓存</category>
|
|
<category>穿透</category>
|
|
<category>击穿</category>
|
|
<category>雪崩</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Redis</tag>
|
|
<tag>缓存穿透</tag>
|
|
<tag>缓存击穿</tag>
|
|
<tag>缓存雪崩</tag>
|
|
<tag>布隆过滤器</tag>
|
|
<tag>bloom filter</tag>
|
|
<tag>互斥锁</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊Java中的单例模式</title>
|
|
<url>/2019/12/21/%E8%81%8A%E8%81%8AJava%E4%B8%AD%E7%9A%84%E5%8D%95%E4%BE%8B%E6%A8%A1%E5%BC%8F/</url>
|
|
<content><![CDATA[<p>这是个 Java 面试的高频问题，我也遇到过，以往都是觉得这类题没意思，网上一搜一大堆，也不愿意记，其实说回来，主要还是没静下心来好好去理解，今天无意中看到一个课程，基本帮我把一些疑惑的点讲清楚了，首先单例是啥意思，这个其实是有范围一说，比如我起了个<code>Spring Boot</code>应用，在这个应用范围内，我的常规 bean 是单例的，意味着 getBean 的时候其实永远只会拿到那一个对象，那要怎么来写一个单例呢，首先就是传说中的饿汉模式，也是最简单的</p>
|
|
<h2 id="饿汉模式"><a href="#饿汉模式" class="headerlink" title="饿汉模式"></a>饿汉模式</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">Singleton1</span> {</span><br><span class="line"> <span class="comment">// 首先，将构造方法变成私有的</span></span><br><span class="line"> <span class="keyword">private</span> <span class="title function_">Singleton1</span><span class="params">()</span> {};</span><br><span class="line"> <span class="comment">// 创建私有静态实例，这样第一次使用的时候就会进行创建</span></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="type">Singleton</span> <span class="variable">instance</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Singleton1</span>();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 使用这个对象都是通过这个 getInstance 来获取</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> Singleton1 <span class="title function_">getInstance</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> instance;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 瞎写一个静态方法。这里想说的是，如果我们只是要调用 Singleton.getDate(...)，</span></span><br><span class="line"> <span class="comment">// 本来是不想要生成 Singleton 实例的，不过没办法，已经生成了</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> Date <span class="title function_">getDate</span><span class="params">(String mode)</span> {<span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">Date</span>();}</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>上面借鉴了一些代码，其实这是最基本，也不会错的方法，但是正如其中<code>getDate</code>方法里说的问题，有时候并没有想那这个对象，但是因为我调用了这个类的静态方法，导致对象已经生成了，可能这也是饿汉模式名字的来由，不管三七二十一给你生成个单例就完事了，不管有没有用，但是这种个人觉得也没啥大问题，如果是面试的话最好说出来它的缺点</p>
|
|
<h2 id="饱汉模式"><a href="#饱汉模式" class="headerlink" title="饱汉模式"></a>饱汉模式</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">Singleton2</span> {</span><br><span class="line"> <span class="comment">// 首先，也是先堵死 new Singleton() 这条路,将构造方法变成私有</span></span><br><span class="line"> <span class="keyword">private</span> <span class="title function_">Singleton2</span><span class="params">()</span> {}</span><br><span class="line"> <span class="comment">// 和饿汉模式相比，这边不需要先实例化出来，注意这里的 volatile，它是必须的</span></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">volatile</span> <span class="type">Singleton2</span> <span class="variable">instance</span> <span class="operator">=</span> <span class="literal">null</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="type">int</span> <span class="variable">m</span> <span class="operator">=</span> <span class="number">9</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> Singleton <span class="title function_">getInstance</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">if</span> (instance == <span class="literal">null</span>) {</span><br><span class="line"> <span class="comment">// 加锁</span></span><br><span class="line"> <span class="keyword">synchronized</span> (Singleton2.class) {</span><br><span class="line"> <span class="comment">// 这一次判断也是必须的，不然会有并发问题</span></span><br><span class="line"> <span class="keyword">if</span> (instance == <span class="literal">null</span>) {</span><br><span class="line"> instance = <span class="keyword">new</span> <span class="title class_">Singleton2</span>();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> instance;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里容易错的有三点，理解了其实就比较好记了</p>
|
|
<p>第一点，为啥不在 getInstance 上整个代码块加 <code>synchronized</code>，这个其实比较容易理解，就是锁的力度太大，性能太差了，这点其实也要去理解，可以举个夸张的例子，比如我一个电商的服务，如果为了避免一个人的订单出现问题，是不是可以从请求入口就把他锁住，到请求结束释放，那么里面做的事情都有保障，然而这显然不可能，因为我们想要这种竞态条件抢占资源的时间尽量减少，防止其他线程等待。<br>第二点，为啥<code>synchronized</code>之已经检查了 <code>instance == null</code>,还要在里面再检查一次，这个有个术语，叫 <code>double check lock</code>,但是为啥要这么做呢，其实很简单，想象当有两个线程，都过了第一步为空判断，这个时候只有一个线程能拿到这个锁，另一个线程就等待了，如果不再判断一次，那么第一个线程新建完对象释放锁之后，第二个线程又能拿到锁，再去创建一个对象。<br>第三点，为啥要<code>volatile</code>关键字，原先对它的理解是它修饰的变量在 JMM 中能及时将变量值写到主存中，但是它还有个很重要的作用，就是防止指令重排序，<code>instance = new Singleton();</code>这行代码其实在底层是分成三条指令执行的，第一条是在堆上申请了一块内存放这个对象，但是对象的字段啥的都还是默认值，第二条是设置对象的值，比如上面的 m 是 9，然后第三条是将这个对象和虚拟机栈上的指针建立引用关联，那么如果我不用<code>volatile</code>关键字，这三条指令就有可能出现重排，比如变成了 1-3-2 这种顺序，当执行完第二步时，有个线程来访问这个对象了，先判断是不是空，发现不是空的，就拿去直接用了，是不是就出现问题了，所以这个<code>volatile</code>也是不可缺少的</p>
|
|
<h2 id="嵌套类"><a href="#嵌套类" class="headerlink" title="嵌套类"></a>嵌套类</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">Singleton3</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="title function_">Singleton3</span><span class="params">()</span> {}</span><br><span class="line"> <span class="comment">// 主要是使用了 嵌套类可以访问外部类的静态属性和静态方法 的特性</span></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">class</span> <span class="title class_">Holder</span> {</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="type">Singleton3</span> <span class="variable">instance</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Singleton3</span>();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> Singleton3 <span class="title function_">getInstance</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> Holder.instance;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这个我个人感觉是饿汉模式的升级版，可以在调用<code>getInstance</code>的时候去实例化对象，也是比较推荐的</p>
|
|
<h2 id="枚举单例"><a href="#枚举单例" class="headerlink" title="枚举单例"></a>枚举单例</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">enum</span> <span class="title class_">Singleton</span> {</span><br><span class="line"> INSTANCE;</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">doSomething</span><span class="params">()</span>{</span><br><span class="line"> <span class="comment">//todo doSomething</span></span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>枚举很特殊，它在类加载的时候会初始化里面的所有的实例，而且 JVM 保证了它们不会再被实例化，所以它天生就是单例的。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Design Patterns</category>
|
|
<category>Singleton</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>设计模式</tag>
|
|
<tag>Design Patterns</tag>
|
|
<tag>单例</tag>
|
|
<tag>Singleton</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊 SpringBoot 自动装配</title>
|
|
<url>/2021/07/11/%E8%81%8A%E8%81%8ASpringBoot-%E8%87%AA%E5%8A%A8%E8%A3%85%E9%85%8D/</url>
|
|
<content><![CDATA[<p>springboot 自动装配调用链</p>
|
|
<p>springboot 相比 spring能更方便开发人员上手，比较重要的一点就是自动装配，大致来看下这个逻辑</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> SpringApplication.run(SpbDemoApplication.class, args);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Static helper that can be used to run a {<span class="doctag">@link</span> SpringApplication} from the</span></span><br><span class="line"><span class="comment"> * specified source using default settings.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> primarySource the primary source to load</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> args the application arguments (usually passed from a Java main method)</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> the running {<span class="doctag">@link</span> ApplicationContext}</span></span><br><span class="line"><span class="comment"> */</span></span><br></pre></td></tr></table></figure>
|
|
<p>然后就是上面调用的 run 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> ConfigurableApplicationContext <span class="title function_">run</span><span class="params">(Class<?> primarySource, String... args)</span> {</span><br><span class="line"> <span class="keyword">return</span> run(<span class="keyword">new</span> <span class="title class_">Class</span><?>[] { primarySource }, args);</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Static helper that can be used to run a {<span class="doctag">@link</span> SpringApplication} from the</span></span><br><span class="line"><span class="comment"> * specified sources using default settings and user supplied arguments.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> primarySources the primary sources to load</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> args the application arguments (usually passed from a Java main method)</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> the running {<span class="doctag">@link</span> ApplicationContext}</span></span><br><span class="line"><span class="comment"> */</span></span><br></pre></td></tr></table></figure>
|
|
<p>继续往下看</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> ConfigurableApplicationContext <span class="title function_">run</span><span class="params">(Class<?>[] primarySources, String[] args)</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">SpringApplication</span>(primarySources).run(args);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<span id="more"></span>
|
|
<p>调用SpringApplication的构造方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Create a new {<span class="doctag">@link</span> SpringApplication} instance. The application context will load</span></span><br><span class="line"><span class="comment"> * beans from the specified primary sources (see {<span class="doctag">@link</span> SpringApplication class-level}</span></span><br><span class="line"><span class="comment"> * documentation for details. The instance can be customized before calling</span></span><br><span class="line"><span class="comment"> * {<span class="doctag">@link</span> #run(String...)}.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> primarySources the primary bean sources</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> #run(Class, String[])</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> #SpringApplication(ResourceLoader, Class...)</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> #setSources(Set)</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="title function_">SpringApplication</span><span class="params">(Class<?>... primarySources)</span> {</span><br><span class="line"> <span class="built_in">this</span>(<span class="literal">null</span>, primarySources);</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Create a new {<span class="doctag">@link</span> SpringApplication} instance. The application context will load</span></span><br><span class="line"><span class="comment"> * beans from the specified primary sources (see {<span class="doctag">@link</span> SpringApplication class-level}</span></span><br><span class="line"><span class="comment"> * documentation for details. The instance can be customized before calling</span></span><br><span class="line"><span class="comment"> * {<span class="doctag">@link</span> #run(String...)}.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> resourceLoader the resource loader to use</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> primarySources the primary bean sources</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> #run(Class, String[])</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> #setSources(Set)</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="meta">@SuppressWarnings({ "unchecked", "rawtypes" })</span></span><br><span class="line"><span class="keyword">public</span> <span class="title function_">SpringApplication</span><span class="params">(ResourceLoader resourceLoader, Class<?>... primarySources)</span> {</span><br><span class="line"> <span class="built_in">this</span>.resourceLoader = resourceLoader;</span><br><span class="line"> Assert.notNull(primarySources, <span class="string">"PrimarySources must not be null"</span>);</span><br><span class="line"> <span class="built_in">this</span>.primarySources = <span class="keyword">new</span> <span class="title class_">LinkedHashSet</span><>(Arrays.asList(primarySources));</span><br><span class="line"> <span class="built_in">this</span>.webApplicationType = WebApplicationType.deduceFromClasspath();</span><br><span class="line"> <span class="comment">// 注意看这里的，通过 SpringFactories 获取</span></span><br><span class="line"> <span class="built_in">this</span>.bootstrapRegistryInitializers = getBootstrapRegistryInitializersFromSpringFactories();</span><br><span class="line"> setInitializers((Collection) getSpringFactoriesInstances(ApplicationContextInitializer.class));</span><br><span class="line"> setListeners((Collection) getSpringFactoriesInstances(ApplicationListener.class));</span><br><span class="line"> <span class="built_in">this</span>.mainApplicationClass = deduceMainApplicationClass();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里就是重点了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> List<BootstrapRegistryInitializer> <span class="title function_">getBootstrapRegistryInitializersFromSpringFactories</span><span class="params">()</span> {</span><br><span class="line"> ArrayList<BootstrapRegistryInitializer> initializers = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"> getSpringFactoriesInstances(Bootstrapper.class).stream()</span><br><span class="line"> .map((bootstrapper) -> ((BootstrapRegistryInitializer) bootstrapper::initialize))</span><br><span class="line"> .forEach(initializers::add);</span><br><span class="line"> initializers.addAll(getSpringFactoriesInstances(BootstrapRegistryInitializer.class));</span><br><span class="line"> <span class="keyword">return</span> initializers;</span><br><span class="line">}</span><br><span class="line"><span class="keyword">private</span> <T> Collection<T> <span class="title function_">getSpringFactoriesInstances</span><span class="params">(Class<T> type)</span> {</span><br><span class="line"> <span class="keyword">return</span> getSpringFactoriesInstances(type, <span class="keyword">new</span> <span class="title class_">Class</span><?>[] {});</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">private</span> <T> Collection<T> <span class="title function_">getSpringFactoriesInstances</span><span class="params">(Class<T> type, Class<?>[] parameterTypes, Object... args)</span> {</span><br><span class="line"> <span class="type">ClassLoader</span> <span class="variable">classLoader</span> <span class="operator">=</span> getClassLoader();</span><br><span class="line"> <span class="comment">// Use names and ensure unique to protect against duplicates</span></span><br><span class="line"> <span class="comment">// 去加载所有FACTORIES_RESOURCE_LOCATION路径下面，也就是 META-INF/spring.factories</span></span><br><span class="line"> Set<String> names = <span class="keyword">new</span> <span class="title class_">LinkedHashSet</span><>(SpringFactoriesLoader.loadFactoryNames(type, classLoader));</span><br><span class="line"> List<T> instances = createSpringFactoriesInstances(type, parameterTypes, classLoader, args, names);</span><br><span class="line"> AnnotationAwareOrderComparator.sort(instances);</span><br><span class="line"> <span class="keyword">return</span> instances;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Load the fully qualified class names of factory implementations of the</span></span><br><span class="line"><span class="comment"> * given type from {<span class="doctag">@value</span> #FACTORIES_RESOURCE_LOCATION}, using the given</span></span><br><span class="line"><span class="comment"> * class loader.</span></span><br><span class="line"><span class="comment"> * <p>As of Spring Framework 5.3, if a particular implementation class name</span></span><br><span class="line"><span class="comment"> * is discovered more than once for the given factory type, duplicates will</span></span><br><span class="line"><span class="comment"> * be ignored.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> factoryType the interface or abstract class representing the factory</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> classLoader the ClassLoader to use for loading resources; can be</span></span><br><span class="line"><span class="comment"> * {<span class="doctag">@code</span> null} to use the default</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@throws</span> IllegalArgumentException if an error occurs while loading factory names</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> #loadFactories</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> List<String> <span class="title function_">loadFactoryNames</span><span class="params">(Class<?> factoryType, <span class="meta">@Nullable</span> ClassLoader classLoader)</span> {</span><br><span class="line"> <span class="type">ClassLoader</span> <span class="variable">classLoaderToUse</span> <span class="operator">=</span> classLoader;</span><br><span class="line"> <span class="keyword">if</span> (classLoaderToUse == <span class="literal">null</span>) {</span><br><span class="line"> classLoaderToUse = SpringFactoriesLoader.class.getClassLoader();</span><br><span class="line"> }</span><br><span class="line"> <span class="type">String</span> <span class="variable">factoryTypeName</span> <span class="operator">=</span> factoryType.getName();</span><br><span class="line"> <span class="keyword">return</span> loadSpringFactories(classLoaderToUse).getOrDefault(factoryTypeName, Collections.emptyList());</span><br><span class="line">}</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> Map<String, List<String>> <span class="title function_">loadSpringFactories</span><span class="params">(ClassLoader classLoader)</span> {</span><br><span class="line"> Map<String, List<String>> result = cache.get(classLoader);</span><br><span class="line"> <span class="keyword">if</span> (result != <span class="literal">null</span>) {</span><br><span class="line"> <span class="keyword">return</span> result;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> result = <span class="keyword">new</span> <span class="title class_">HashMap</span><>();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 获取此 resources，作为 AutoConfiguration 的配置</span></span><br><span class="line"> Enumeration<URL> urls = classLoader.getResources(FACTORIES_RESOURCE_LOCATION);</span><br><span class="line"> <span class="keyword">while</span> (urls.hasMoreElements()) {</span><br><span class="line"> <span class="type">URL</span> <span class="variable">url</span> <span class="operator">=</span> urls.nextElement();</span><br><span class="line"> <span class="type">UrlResource</span> <span class="variable">resource</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">UrlResource</span>(url);</span><br><span class="line"> <span class="type">Properties</span> <span class="variable">properties</span> <span class="operator">=</span> PropertiesLoaderUtils.loadProperties(resource);</span><br><span class="line"> <span class="keyword">for</span> (Map.Entry<?, ?> entry : properties.entrySet()) {</span><br><span class="line"> <span class="type">String</span> <span class="variable">factoryTypeName</span> <span class="operator">=</span> ((String) entry.getKey()).trim();</span><br><span class="line"> String[] factoryImplementationNames =</span><br><span class="line"> StringUtils.commaDelimitedListToStringArray((String) entry.getValue());</span><br><span class="line"> <span class="keyword">for</span> (String factoryImplementationName : factoryImplementationNames) {</span><br><span class="line"> result.computeIfAbsent(factoryTypeName, key -> <span class="keyword">new</span> <span class="title class_">ArrayList</span><>())</span><br><span class="line"> .add(factoryImplementationName.trim());</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Replace all lists with unmodifiable lists containing unique elements</span></span><br><span class="line"> result.replaceAll((factoryType, implementations) -> implementations.stream().distinct()</span><br><span class="line"> .collect(Collectors.collectingAndThen(Collectors.toList(), Collections::unmodifiableList)));</span><br><span class="line"> cache.put(classLoader, result);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">catch</span> (IOException ex) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalArgumentException</span>(<span class="string">"Unable to load factories from location ["</span> +</span><br><span class="line"> FACTORIES_RESOURCE_LOCATION + <span class="string">"]"</span>, ex);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> result;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>我们可以看下 spring-boot-autoconfigure 的 META-INF/spring.factories</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"># Initializers</span><br><span class="line">org.springframework.context.ApplicationContextInitializer=\</span><br><span class="line">org.springframework.boot.autoconfigure.SharedMetadataReaderFactoryContextInitializer,\</span><br><span class="line">org.springframework.boot.autoconfigure.logging.ConditionEvaluationReportLoggingListener</span><br><span class="line"></span><br><span class="line"># Application Listeners</span><br><span class="line">org.springframework.context.ApplicationListener=\</span><br><span class="line">org.springframework.boot.autoconfigure.BackgroundPreinitializer</span><br><span class="line"></span><br><span class="line"># Environment Post Processors</span><br><span class="line">org.springframework.boot.env.EnvironmentPostProcessor=\</span><br><span class="line">org.springframework.boot.autoconfigure.integration.IntegrationPropertiesEnvironmentPostProcessor</span><br><span class="line"></span><br><span class="line"># Auto Configuration Import Listeners</span><br><span class="line">org.springframework.boot.autoconfigure.AutoConfigurationImportListener=\</span><br><span class="line">org.springframework.boot.autoconfigure.condition.ConditionEvaluationReportAutoConfigurationImportListener</span><br><span class="line"></span><br><span class="line"># Auto Configuration Import Filters</span><br><span class="line">org.springframework.boot.autoconfigure.AutoConfigurationImportFilter=\</span><br><span class="line">org.springframework.boot.autoconfigure.condition.OnBeanCondition,\</span><br><span class="line">org.springframework.boot.autoconfigure.condition.OnClassCondition,\</span><br><span class="line">org.springframework.boot.autoconfigure.condition.OnWebApplicationCondition</span><br><span class="line"></span><br><span class="line"># 注意这里，其实就是类似于 dubbo spi 的通过 org.springframework.boot.autoconfigure.EnableAutoConfiguration作为 key</span><br><span class="line"># 获取下面所有的 AutoConfiguration 配置类</span><br><span class="line"># Auto Configure</span><br><span class="line">org.springframework.boot.autoconfigure.EnableAutoConfiguration=\</span><br><span class="line">org.springframework.boot.autoconfigure.admin.SpringApplicationAdminJmxAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.aop.AopAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.amqp.RabbitAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.batch.BatchAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.cache.CacheAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.cassandra.CassandraAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.context.ConfigurationPropertiesAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.context.LifecycleAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.context.MessageSourceAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.context.PropertyPlaceholderAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.couchbase.CouchbaseAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.dao.PersistenceExceptionTranslationAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.cassandra.CassandraDataAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.cassandra.CassandraReactiveDataAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.cassandra.CassandraReactiveRepositoriesAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.cassandra.CassandraRepositoriesAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.couchbase.CouchbaseDataAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.couchbase.CouchbaseReactiveDataAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.couchbase.CouchbaseReactiveRepositoriesAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.couchbase.CouchbaseRepositoriesAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.elasticsearch.ElasticsearchDataAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.elasticsearch.ElasticsearchRepositoriesAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.elasticsearch.ReactiveElasticsearchRepositoriesAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.elasticsearch.ReactiveElasticsearchRestClientAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.jdbc.JdbcRepositoriesAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.jpa.JpaRepositoriesAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.ldap.LdapRepositoriesAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.mongo.MongoDataAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.mongo.MongoReactiveDataAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.mongo.MongoReactiveRepositoriesAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.mongo.MongoRepositoriesAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.neo4j.Neo4jDataAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.neo4j.Neo4jReactiveDataAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.neo4j.Neo4jReactiveRepositoriesAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.neo4j.Neo4jRepositoriesAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.r2dbc.R2dbcDataAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.r2dbc.R2dbcRepositoriesAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.redis.RedisAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.redis.RedisReactiveAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.redis.RedisRepositoriesAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.rest.RepositoryRestMvcAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.data.web.SpringDataWebAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.elasticsearch.ElasticsearchRestClientAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.flyway.FlywayAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.freemarker.FreeMarkerAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.groovy.template.GroovyTemplateAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.gson.GsonAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.h2.H2ConsoleAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.hateoas.HypermediaAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.hazelcast.HazelcastAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.hazelcast.HazelcastJpaDependencyAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.http.HttpMessageConvertersAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.http.codec.CodecsAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.influx.InfluxDbAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.info.ProjectInfoAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.integration.IntegrationAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.jackson.JacksonAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.jdbc.JdbcTemplateAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.jdbc.JndiDataSourceAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.jdbc.XADataSourceAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.jdbc.DataSourceTransactionManagerAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.jms.JmsAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.jmx.JmxAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.jms.JndiConnectionFactoryAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.jms.activemq.ActiveMQAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.jms.artemis.ArtemisAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.jersey.JerseyAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.jooq.JooqAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.jsonb.JsonbAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.kafka.KafkaAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.availability.ApplicationAvailabilityAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.ldap.embedded.EmbeddedLdapAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.ldap.LdapAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.liquibase.LiquibaseAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.mail.MailSenderAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.mail.MailSenderValidatorAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.mongo.embedded.EmbeddedMongoAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.mongo.MongoReactiveAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.mustache.MustacheAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.neo4j.Neo4jAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.netty.NettyAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.quartz.QuartzAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.r2dbc.R2dbcAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.r2dbc.R2dbcTransactionManagerAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.rsocket.RSocketMessagingAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.rsocket.RSocketRequesterAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.rsocket.RSocketServerAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.rsocket.RSocketStrategiesAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.security.servlet.SecurityAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.security.servlet.UserDetailsServiceAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.security.servlet.SecurityFilterAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.security.reactive.ReactiveSecurityAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.security.reactive.ReactiveUserDetailsServiceAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.security.rsocket.RSocketSecurityAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.security.saml2.Saml2RelyingPartyAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.sendgrid.SendGridAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.session.SessionAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.security.oauth2.client.servlet.OAuth2ClientAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.security.oauth2.client.reactive.ReactiveOAuth2ClientAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.security.oauth2.resource.servlet.OAuth2ResourceServerAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.security.oauth2.resource.reactive.ReactiveOAuth2ResourceServerAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.solr.SolrAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.sql.init.SqlInitializationAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.task.TaskExecutionAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.task.TaskSchedulingAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.thymeleaf.ThymeleafAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.transaction.TransactionAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.transaction.jta.JtaAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.validation.ValidationAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.web.client.RestTemplateAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.web.embedded.EmbeddedWebServerFactoryCustomizerAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.web.reactive.HttpHandlerAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.web.reactive.ReactiveWebServerFactoryAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.web.reactive.WebFluxAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.web.reactive.error.ErrorWebFluxAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.web.reactive.function.client.ClientHttpConnectorAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.web.reactive.function.client.WebClientAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.web.servlet.DispatcherServletAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.web.servlet.ServletWebServerFactoryAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.web.servlet.error.ErrorMvcAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.web.servlet.HttpEncodingAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.web.servlet.MultipartAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.web.servlet.WebMvcAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.websocket.reactive.WebSocketReactiveAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.websocket.servlet.WebSocketServletAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.websocket.servlet.WebSocketMessagingAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.webservices.WebServicesAutoConfiguration,\</span><br><span class="line">org.springframework.boot.autoconfigure.webservices.client.WebServiceTemplateAutoConfiguration</span><br><span class="line"></span><br><span class="line"># Failure analyzers</span><br><span class="line">org.springframework.boot.diagnostics.FailureAnalyzer=\</span><br><span class="line">org.springframework.boot.autoconfigure.data.redis.RedisUrlSyntaxFailureAnalyzer,\</span><br><span class="line">org.springframework.boot.autoconfigure.diagnostics.analyzer.NoSuchBeanDefinitionFailureAnalyzer,\</span><br><span class="line">org.springframework.boot.autoconfigure.flyway.FlywayMigrationScriptMissingFailureAnalyzer,\</span><br><span class="line">org.springframework.boot.autoconfigure.jdbc.DataSourceBeanCreationFailureAnalyzer,\</span><br><span class="line">org.springframework.boot.autoconfigure.jdbc.HikariDriverConfigurationFailureAnalyzer,\</span><br><span class="line">org.springframework.boot.autoconfigure.r2dbc.ConnectionFactoryBeanCreationFailureAnalyzer,\</span><br><span class="line">org.springframework.boot.autoconfigure.session.NonUniqueSessionRepositoryFailureAnalyzer</span><br><span class="line"></span><br><span class="line"># Template availability providers</span><br><span class="line">org.springframework.boot.autoconfigure.template.TemplateAvailabilityProvider=\</span><br><span class="line">org.springframework.boot.autoconfigure.freemarker.FreeMarkerTemplateAvailabilityProvider,\</span><br><span class="line">org.springframework.boot.autoconfigure.mustache.MustacheTemplateAvailabilityProvider,\</span><br><span class="line">org.springframework.boot.autoconfigure.groovy.template.GroovyTemplateAvailabilityProvider,\</span><br><span class="line">org.springframework.boot.autoconfigure.thymeleaf.ThymeleafTemplateAvailabilityProvider,\</span><br><span class="line">org.springframework.boot.autoconfigure.web.servlet.JspTemplateAvailabilityProvider</span><br><span class="line"></span><br><span class="line"># DataSource initializer detectors</span><br><span class="line">org.springframework.boot.sql.init.dependency.DatabaseInitializerDetector=\</span><br><span class="line">org.springframework.boot.autoconfigure.flyway.FlywayMigrationInitializerDatabaseInitializerDetector</span><br><span class="line"></span><br></pre></td></tr></table></figure>
|
|
|
|
<p>上面根据 org.springframework.boot.autoconfigure.EnableAutoConfiguration 获取的各个配置类，在通过反射加载就能得到一堆 JavaConfig配置类，然后再根据 ConditionalOnProperty等条件配置加载具体的 bean，大致就是这么个逻辑</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>SpringBoot</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Spring</tag>
|
|
<tag>SpringBoot</tag>
|
|
<tag>自动装配</tag>
|
|
<tag>AutoConfiguration</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊一聊 https 的逻辑</title>
|
|
<url>/2024/03/10/%E8%81%8A%E4%B8%80%E8%81%8A-https-%E7%9A%84%E9%80%BB%E8%BE%91/</url>
|
|
<content><![CDATA[<h1 id="https-和-tls"><a href="#https-和-tls" class="headerlink" title="https 和 tls"></a>https 和 tls</h1><p>https 我原本的理解来自于几个概念</p>
|
|
<p>第一个是非对称加密</p>
|
|
<p>也就是公钥可以公开在网上，然后用公钥加密，由私有的私钥进行解密，然后就对 https 有了初步的偏颇的概念，就是浏览器用公钥进行加密，服务器端用私钥进行解密，这里再仔细想想会延伸出来几个问题，也是逐步完善我的后续理解</p>
|
|
<p>第二个就是非对称加密以后的对称加密</p>
|
|
<p>因为对加密的理解是超大的质数（这个还来源于大学的数据结构老师）相乘后会很难做质因数分解，比如最大的质数<strong>2^82,589,933 − 1，</strong>用十进制表示有24,862,048位，用现在计算机来算也是非常难的，那如果一直采用非对称加密，这个效率也会比较低，所以在网上查了一番之后知道了是先采用非对称加密，然后再用对称加密，因为非对称加密先有了安全保证，后续的消息就可以用对称加密的方式来进行传输安全了</p>
|
|
<p>第三个是最近一次看到的</p>
|
|
<p>也是一直缺乏思考的问题，因为 https 我们在使用的时候都会购买证书或者使用免费的证书，那这个证书和前面说的公私钥是什么关系，好像一直是悬空的，没想过这中间的相关性，证书究竟是怎么起作用的</p>
|
|
<p>下面是摘自 cloudflare 的 tls 握手流程，把证书的使用方式说的比较明白了</p>
|
|
<p>TLS 握手是由客户端和服务器交换的一系列数据报或消息。TLS 握手涉及多个步骤，因为客户端和服务器要交换完成握手和进行进一步对话所需的信息。</p>
|
|
<p>TLS 握手中的确切步骤将根据所使用的密钥交换算法的种类和双方支持的密码套件而有所不同。RSA 密钥交换算法虽然现在被认为不安全，但曾在 1.3 之前的 TLS 版本中使用。大致如下：</p>
|
|
<ol>
|
|
<li><strong>“客户端问候（client hello）” 消息：</strong> 客户端通过向服务器发送“问候”消息来开始握手。该消息将包含客户端支持的 TLS 版本，支持的密码套件，以及称为一串称为“客户端随机数（client random）”的随机字节。</li>
|
|
<li><strong>“服务器问候（server hello）”消息：</strong> 作为对 client hello 消息的回复，服务器发送一条消息，内含服务器的 <a href="https://www.cloudflare.com/learning/ssl/what-is-an-ssl-certificate/">SSL 证书</a>、服务器选择的密码套件，以及“服务器随机数（server random）”，即由服务器生成的另一串随机字节。</li>
|
|
<li><strong>身份验证：</strong> 客户端使用颁发该证书的证书颁发机构验证服务器的 SSL 证书。此举确认服务器是其声称的身份，且客户端正在与该域的实际所有者进行交互。</li>
|
|
<li><strong>预主密钥：</strong> 客户端再发送一串随机字节，即“预主密钥（premaster secret）”。预主密钥是使用公钥加密的，只能使用服务器的私钥解密。（客户端从服务器的 SSL 证书中获得<a href="https://www.cloudflare.com/learning/ssl/how-does-public-key-encryption-work/">公钥</a>。）</li>
|
|
<li><strong>私钥被使用：</strong>服务器对预主密钥进行解密。</li>
|
|
<li><strong>生成会话密钥：</strong>客户端和服务器均使用客户端随机数、服务器随机数和预主密钥生成会话密钥。双方应得到相同的结果。</li>
|
|
<li><strong>客户端就绪：</strong>客户端发送一条“已完成”消息，该消息用会话密钥加密。</li>
|
|
<li><strong>服务器就绪：</strong>服务器发送一条“已完成”消息，该消息用会话密钥加密。</li>
|
|
<li><strong>实现安全对称加密：</strong>已完成握手，并使用会话密钥继续进行通信。</li>
|
|
</ol>
|
|
<p>为了学习这个过程，我们尝试用 Chrome 的自带抓包工具，以往这是可以通过在 Chrome 地址栏中输入 <code>chrome://net-internals/#events</code> 来打开，现在变成了两部分，<code>chrome://net-export/</code> 这里可以开始抓包，然后会记录下一个抓包日志文件里，</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/YpEvbB.png"></p>
|
|
<p>然后再打开</p>
|
|
<p><code>https://netlog-viewer.appspot.com/#events</code> 来查看具体的日志</p>
|
|
<p>这里我们以打开百度为例，即在打开 <code>chrome://net-export/</code> 并启动抓包后，再在一个新 tab 打开 <code>baidu.com</code>，然后关闭</p>
|
|
<p>将日志文件在 <code>https://netlog-viewer.appspot.com/#events</code> 打开</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/4Mz3t1.png"></p>
|
|
<p>这里可以看到 </p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/boil2F.png"></p>
|
|
<p><code>--> type 1</code> 表示现在 tls 握手进行到哪一步了, 对应的值表示不同的阶段</p>
|
|
<table>
|
|
<thead>
|
|
<tr>
|
|
<th>代码(type)</th>
|
|
<th>释义</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr>
|
|
<td>0</td>
|
|
<td>HelloRequest</td>
|
|
</tr>
|
|
<tr>
|
|
<td>1</td>
|
|
<td>ClientHello</td>
|
|
</tr>
|
|
<tr>
|
|
<td>2</td>
|
|
<td>ServerHello</td>
|
|
</tr>
|
|
<tr>
|
|
<td>11</td>
|
|
<td>Certificate</td>
|
|
</tr>
|
|
<tr>
|
|
<td>12</td>
|
|
<td>ServerKeyExchange</td>
|
|
</tr>
|
|
<tr>
|
|
<td>13</td>
|
|
<td>CertificateRequest</td>
|
|
</tr>
|
|
<tr>
|
|
<td>14</td>
|
|
<td>ServerHelloDone</td>
|
|
</tr>
|
|
<tr>
|
|
<td>15</td>
|
|
<td>CertificateVerify</td>
|
|
</tr>
|
|
<tr>
|
|
<td>16</td>
|
|
<td>ClientKeyExchange</td>
|
|
</tr>
|
|
<tr>
|
|
<td>20</td>
|
|
<td>Finished</td>
|
|
</tr>
|
|
</tbody></table>
|
|
<p>具体的对应关系是在这边</p>
|
|
<p>比如 11 之后表示我们从服务端收到了证书</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/K3so7R.png"></p>
|
|
<p>然后就要去验证证书的可靠性</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/4git0i.png"></p>
|
|
<p>后面是验证后的结果</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/1Wt4mU.png"></p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/kULsYB.png"></p>
|
|
<p>最后就是交换对称会话秘钥然后完成握手</p>
|
|
<p><img data-src="https://img.nicksxs.me/blog/sy7meM.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>网络</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>http</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊一下 Spring 的 SmartLifecycle 使用</title>
|
|
<url>/2024/07/07/%E8%81%8A%E4%B8%80%E4%B8%8B-Spring-%E7%9A%84-SmarLifecycle-%E4%BD%BF%E7%94%A8/</url>
|
|
<content><![CDATA[<p>最近在学习过程中碰到一个比较有意思的Spring特性，就是 SmartLifecycle ，这个可以很轻松的融合进 Spring 生命周期</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">interface</span> <span class="title class_">SmartLifecycle</span> <span class="keyword">extends</span> <span class="title class_">Lifecycle</span>, Phased {</span><br><span class="line"> <span class="type">int</span> <span class="variable">DEFAULT_PHASE</span> <span class="operator">=</span> Integer.MAX_VALUE;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">default</span> <span class="type">boolean</span> <span class="title function_">isAutoStartup</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">default</span> <span class="keyword">void</span> <span class="title function_">stop</span><span class="params">(Runnable callback)</span> {</span><br><span class="line"> <span class="built_in">this</span>.stop();</span><br><span class="line"> callback.run();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">default</span> <span class="type">int</span> <span class="title function_">getPhase</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> Integer.MAX_VALUE;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>接口继承了 <code>org.springframework.context.Lifecycle</code> 跟 <code>org.springframework.context.Phased</code><br>其中默认实现了 <code>isAutoStartup</code> , 并且默认是返回 <code>true</code>，所以相对于 <code>Lifecycle</code> 能更智能些自动启动<br>然后 Lifecycle 这个接口就比较简单，只有几个接口</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">interface</span> <span class="title class_">Lifecycle</span> {</span><br><span class="line"> <span class="keyword">void</span> <span class="title function_">start</span><span class="params">()</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">void</span> <span class="title function_">stop</span><span class="params">()</span>;</span><br><span class="line"></span><br><span class="line"> <span class="type">boolean</span> <span class="title function_">isRunning</span><span class="params">()</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>我们可以是实现下这个接口</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Component</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">MySmartLifecycle</span> <span class="keyword">implements</span> <span class="title class_">SmartLifecycle</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">volatile</span> <span class="type">boolean</span> isRunning;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">start</span><span class="params">()</span> {</span><br><span class="line"> System.out.println(<span class="string">"my smart life cycle start"</span>);</span><br><span class="line"> isRunning = <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">stop</span><span class="params">()</span> {</span><br><span class="line"> System.out.println(<span class="string">"my smart life cycle end"</span>);</span><br><span class="line"> isRunning = <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="type">boolean</span> <span class="title function_">isRunning</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> isRunning;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这个 <code>bean</code> 会在 <code>Spring</code> 启动后被自动调用 <code>start</code> 方法<br><img data-src="https://img.nicksxs.me/blog/jbDyzo.png"><br>这个就是在不深入 <code>bean</code> 的生命周期，并且是在 <code>bean</code> 已经都初始化以后调用<br>同样会在 <code>spring</code> 容器关闭时调用 <code>stop</code> 方法<br><img data-src="https://img.nicksxs.me/blog/QPM6wi.png"><br>如果有此类的需求就可以扩展此接口实现<br>而这个其实是在 spring 生命周期中的 finishRefresh 方法中进行调用</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title function_">finishRefresh</span><span class="params">()</span> {</span><br><span class="line"> <span class="built_in">this</span>.clearResourceCaches();</span><br><span class="line"> <span class="built_in">this</span>.initLifecycleProcessor();</span><br><span class="line"> <span class="built_in">this</span>.getLifecycleProcessor().onRefresh();</span><br><span class="line"> <span class="built_in">this</span>.publishEvent((ApplicationEvent)(<span class="keyword">new</span> <span class="title class_">ContextRefreshedEvent</span>(<span class="built_in">this</span>)));</span><br><span class="line"> <span class="keyword">if</span> (!NativeDetector.inNativeImage()) {</span><br><span class="line"> LiveBeansView.registerApplicationContext(<span class="built_in">this</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>调用了 <code>LifecycleProcessor</code> 的 <code>onRefresh</code> 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">onRefresh</span><span class="params">()</span> {</span><br><span class="line"> <span class="built_in">this</span>.startBeans(<span class="literal">true</span>);</span><br><span class="line"> <span class="built_in">this</span>.running = <span class="literal">true</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这其中就会调用 <code>start</code> 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">startBeans</span><span class="params">(<span class="type">boolean</span> autoStartupOnly)</span> {</span><br><span class="line"> <span class="comment">// 获取这些bean</span></span><br><span class="line"> Map<String, Lifecycle> lifecycleBeans = <span class="built_in">this</span>.getLifecycleBeans();</span><br><span class="line"> Map<Integer, LifecycleGroup> phases = <span class="keyword">new</span> <span class="title class_">TreeMap</span>();</span><br><span class="line"> lifecycleBeans.forEach((beanName, bean) -> {</span><br><span class="line"> <span class="keyword">if</span> (!autoStartupOnly || bean <span class="keyword">instanceof</span> SmartLifecycle && ((SmartLifecycle)bean).isAutoStartup()) {</span><br><span class="line"> <span class="type">int</span> <span class="variable">phase</span> <span class="operator">=</span> <span class="built_in">this</span>.getPhase(bean);</span><br><span class="line"> ((LifecycleGroup)phases.computeIfAbsent(phase, (p) -> {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">LifecycleGroup</span>(phase, <span class="built_in">this</span>.timeoutPerShutdownPhase, lifecycleBeans, autoStartupOnly);</span><br><span class="line"> })).add(beanName, bean);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> });</span><br><span class="line"> <span class="keyword">if</span> (!phases.isEmpty()) {</span><br><span class="line"> phases.values().forEach(LifecycleGroup::start);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>会先判断 <code>isAutoStartup</code>，然后将不同周期加入分组，然后再循环调用<code>start</code> </p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊一次 brew update 引发的血案</title>
|
|
<url>/2020/06/13/%E8%81%8A%E8%81%8A%E4%B8%80%E6%AC%A1-brew-update-%E5%BC%95%E5%8F%91%E7%9A%84%E8%A1%80%E6%A1%88/</url>
|
|
<content><![CDATA[<p>熟悉我的人（谁熟悉你啊🙄）知道我以前写过 PHP，虽然现在在工作中没用到了，但是自己的一些小工具还是会用 PHP 来写，但是在 Mac 碰到了一个环境相关的问题，因为我也是个更新狂魔，用了 brew 之后因为 gfw 的原因，如果长时间不更新，有时候要装一个用它装一个软件的话，前置的更新耗时就会让人非常头大，所以我基本会隔天 update 一下，但是这样会带来一个很心烦的问题，就是像这样，因为我是要用一个固定版本的 PHP，如果一直升需要一直配扩展啥的也很麻烦，如果一直升级 PHP 到最新版可能会比较少碰到这个问题</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">dyld: Library not loaded: /usr/local/opt/icu4c/lib/libicui18n.64.dylib</span><br></pre></td></tr></table></figure>
|
|
<p>这是什么鬼啊，然后我去这个目录下看了下，已经都是libicui18n.67.dylib了，而且它没有把原来的版本保留下来，首先这个是个叫 icu4c是啥玩意，谷歌了一下</p>
|
|
<blockquote>
|
|
<p>ICU4C是ICU在C/C++平台下的版本, ICU(International Component for Unicode)是基于”IBM公共许可证”的，与开源组织合作研究的, 用于支持软件国际化的开源项目。ICU4C提供了C/C++平台强大的国际化开发能力，软件开发者几乎可以使用ICU4C解决任何国际化的问题，根据各地的风俗和语言习惯，实现对数字、货币、时间、日期、和消息的格式化、解析，对字符串进行大小写转换、整理、搜索和排序等功能，必须一提的是，ICU4C提供了强大的BIDI算法，对阿拉伯语等BIDI语言提供了完善的支持。</p>
|
|
</blockquote>
|
|
<p>然后首先想到的解决方案就是能不能我使用<code>brew install icu4c@64</code>来重装下原来的版本，发现不行，并木有，之前的做法就只能是去网上把 64 的下载下来，然后放到这个目录，比较麻烦不智能，虽然没抱着希望在谷歌着，不过这次竟然给我找到了一个我认为非常 nice 的解决方案，因为是在 Stack Overflow 找到的，本着写给像我这样的小小白看的，那就稍微翻译一下<br>第一步，我们到 brew的目录下</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">cd $(brew --prefix)/Homebrew/Library/Taps/homebrew/homebrew-core/Formula</span><br></pre></td></tr></table></figure>
|
|
<p>这个可以理解为是 maven 的 pom 文件，不过有很多不同之处，使用ruby 写的，然后一个文件对应一个组件或者软件，那我们看下有个叫icu4c.rb的文件，<br>第二步看看它的提交历史</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">git log --follow icu4c.rb</span><br></pre></td></tr></table></figure>
|
|
<p>在 git log 的海洋中寻找，寻找它的(64版本)的身影<br><img data-src="https://mystore-1255223546.cos.ap-chengdu.myqcloud.com/uPic/YzS7vN.png"><br>第三步注意这三个红框，Stack Overflow 给出来的答案这一步是找到这个 commit id 直接切出一个新分支</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">git checkout -b icu4c-63 e7f0f10dc63b1dc1061d475f1a61d01b70ef2cb7</span><br></pre></td></tr></table></figure>
|
|
<p>其实注意 commit id 旁边的红框，这个是有tag 的，可以直接</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">git checkout icu4c-64</span><br></pre></td></tr></table></figure>
|
|
<p>PS: 因为我的问题是出在 64 的问题，Stack Overflow 回答的是 63 的，反正是一样的解决方法<br>第四部，切回去之后我们就可以用 brew 提供的基于文件的安装命令来重新装上 64 版本</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">brew reinstall ./icu4c.rb</span><br></pre></td></tr></table></figure>
|
|
<p>然后就是第五步，切换版本</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">brew switch icu4c 64.2</span><br></pre></td></tr></table></figure>
|
|
<p>最后把分支切回来</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">git checkout master</span><br></pre></td></tr></table></figure>
|
|
<p>是不是感觉很厉害的解决方法，大佬还提供了一个更牛的，直接写个 zsh 方法</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_"># </span><span class="language-bash">zsh</span></span><br><span class="line">function hiicu64() {</span><br><span class="line"> local last_dir=$(pwd)</span><br><span class="line"></span><br><span class="line"> cd $(brew --prefix)/Homebrew/Library/Taps/homebrew/homebrew-core/Formula</span><br><span class="line"> git checkout icu4c-4</span><br><span class="line"> brew reinstall ./icu4c.rb</span><br><span class="line"> brew switch icu4c 64.2</span><br><span class="line"> git checkout master</span><br><span class="line"></span><br><span class="line"> cd $last_dir</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>对应自己的版本改改版本号就可以了，非常好用。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Mac</category>
|
|
<category>PHP</category>
|
|
<category>Homebrew</category>
|
|
<category>PHP</category>
|
|
<category>icu4c</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Mac</tag>
|
|
<tag>PHP</tag>
|
|
<tag>Homebrew</tag>
|
|
<tag>icu4c</tag>
|
|
<tag>zsh</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊传说中的 ThreadLocal</title>
|
|
<url>/2021/05/30/%E8%81%8A%E8%81%8A%E4%BC%A0%E8%AF%B4%E4%B8%AD%E7%9A%84-ThreadLocal/</url>
|
|
<content><![CDATA[<p>说来也惭愧，这个 ThreadLocal 其实一直都是一知半解，而且看了一下之后还发现记错了，所以还是记录下<br>原先记忆里的都是反过来，一个 ThreadLocal 是里面按照 thread 作为 key，存储线程内容的，真的是半解都米有，完全是错的，这样就得用 concurrentHashMap 这种去存储并且要锁定线程了，然后内容也只能存一个了，想想简直智障</p>
|
|
<h3 id="究竟是啥结构"><a href="#究竟是啥结构" class="headerlink" title="究竟是啥结构"></a>究竟是啥结构</h3><p>比如我们在代码中 new 一个 ThreadLocal，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> ThreadLocal<Man> tl = <span class="keyword">new</span> <span class="title class_">ThreadLocal</span><>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">Thread</span>(() -> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> TimeUnit.SECONDS.sleep(<span class="number">2</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> System.out.println(tl.get());</span><br><span class="line"> }).start();</span><br><span class="line"> <span class="keyword">new</span> <span class="title class_">Thread</span>(() -> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> TimeUnit.SECONDS.sleep(<span class="number">1</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> tl.set(<span class="keyword">new</span> <span class="title class_">Man</span>());</span><br><span class="line"> }).start();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">static</span> <span class="keyword">class</span> <span class="title class_">Man</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">name</span> <span class="operator">=</span> <span class="string">"nick"</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>这里构造了两个线程，一个先往里设值，一个后从里取，运行看下结果，<br><img data-src="https://img.nicksxs.me/uPic/uGpXep.png"><br>知道这个用法的话肯定知道是取不到值的，只是具体的原理原来搞错了，我们来看下设值 set 方法</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">set</span><span class="params">(T value)</span> {</span><br><span class="line"> <span class="type">Thread</span> <span class="variable">t</span> <span class="operator">=</span> Thread.currentThread();</span><br><span class="line"> <span class="type">ThreadLocalMap</span> <span class="variable">map</span> <span class="operator">=</span> getMap(t);</span><br><span class="line"> <span class="keyword">if</span> (map != <span class="literal">null</span>)</span><br><span class="line"> map.set(<span class="built_in">this</span>, value);</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> createMap(t, value);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>写博客这会我才明白我原来咋会错得这么离谱，看到第一行代码 t 就是当前线程，然后第二行就是用这个线程去<code>getMap</code>，然后我是把这个当成从 map 里取值了，其实这里是</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">ThreadLocalMap <span class="title function_">getMap</span><span class="params">(Thread t)</span> {</span><br><span class="line"> <span class="keyword">return</span> t.threadLocals;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>获取 t 的 threadLocals 成员变量，那这个 threadLocals 又是啥呢<br><img data-src="https://img.nicksxs.me/uPic/4p5MdM.png"><br>它其实是线程 Thread 中的一个类型是java.lang.ThreadLocal.ThreadLocalMap的成员变量<br>这是 ThreadLocal 的一个静态成员变量</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">static</span> <span class="keyword">class</span> <span class="title class_">ThreadLocalMap</span> {</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * The entries in this hash map extend WeakReference, using</span></span><br><span class="line"><span class="comment"> * its main ref field as the key (which is always a</span></span><br><span class="line"><span class="comment"> * ThreadLocal object). Note that null keys (i.e. entry.get()</span></span><br><span class="line"><span class="comment"> * == null) mean that the key is no longer referenced, so the</span></span><br><span class="line"><span class="comment"> * entry can be expunged from table. Such entries are referred to</span></span><br><span class="line"><span class="comment"> * as "stale entries" in the code that follows.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">static</span> <span class="keyword">class</span> <span class="title class_">Entry</span> <span class="keyword">extends</span> <span class="title class_">WeakReference</span><ThreadLocal<?>> {</span><br><span class="line"> <span class="comment">/** The value associated with this ThreadLocal. */</span></span><br><span class="line"> Object value;</span><br><span class="line"></span><br><span class="line"> Entry(ThreadLocal<?> k, Object v) {</span><br><span class="line"> <span class="built_in">super</span>(k);</span><br><span class="line"> value = v;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>全部代码有点长，只截取了一小部分，然后我们再回头来分析前面说的 set 过程，再 copy 下代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">set</span><span class="params">(T value)</span> {</span><br><span class="line"> <span class="type">Thread</span> <span class="variable">t</span> <span class="operator">=</span> Thread.currentThread();</span><br><span class="line"> <span class="type">ThreadLocalMap</span> <span class="variable">map</span> <span class="operator">=</span> getMap(t);</span><br><span class="line"> <span class="keyword">if</span> (map != <span class="literal">null</span>)</span><br><span class="line"> map.set(<span class="built_in">this</span>, value);</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> createMap(t, value);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>获取到 map 以后呢，如果 map 不为空，就往 map 里 set，这里注意 key 是啥，其实是当前这个 ThreadLocal，这里就比较明白了究竟是啥结构，每个线程都会维护自身的 ThreadLocalMap，它是线程的一个成员变量，当创建 ThreadLocal 的时候，进行设值的时候其实是往这个 map 里以 ThreadLocal 作为 key，往里设 value。</p>
|
|
<h3 id="内存泄漏是什么鬼"><a href="#内存泄漏是什么鬼" class="headerlink" title="内存泄漏是什么鬼"></a>内存泄漏是什么鬼</h3><p>这里又要看下前面的 ThreadLocalMap 结构了，类似 HashMap，它有个 Entry 结构，在设置的时候会先包装成一个 Entry</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">set</span><span class="params">(ThreadLocal<?> key, Object value)</span> {</span><br><span class="line"></span><br><span class="line"> <span class="comment">// We don't use a fast path as with get() because it is at</span></span><br><span class="line"> <span class="comment">// least as common to use set() to create new entries as</span></span><br><span class="line"> <span class="comment">// it is to replace existing ones, in which case, a fast</span></span><br><span class="line"> <span class="comment">// path would fail more often than not.</span></span><br><span class="line"></span><br><span class="line"> Entry[] tab = table;</span><br><span class="line"> <span class="type">int</span> <span class="variable">len</span> <span class="operator">=</span> tab.length;</span><br><span class="line"> <span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> key.threadLocalHashCode & (len-<span class="number">1</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">Entry</span> <span class="variable">e</span> <span class="operator">=</span> tab[i];</span><br><span class="line"> e != <span class="literal">null</span>;</span><br><span class="line"> e = tab[i = nextIndex(i, len)]) {</span><br><span class="line"> ThreadLocal<?> k = e.get();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (k == key) {</span><br><span class="line"> e.value = value;</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (k == <span class="literal">null</span>) {</span><br><span class="line"> replaceStaleEntry(key, value, i);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> tab[i] = <span class="keyword">new</span> <span class="title class_">Entry</span>(key, value);</span><br><span class="line"> <span class="type">int</span> <span class="variable">sz</span> <span class="operator">=</span> ++size;</span><br><span class="line"> <span class="keyword">if</span> (!cleanSomeSlots(i, sz) && sz >= threshold)</span><br><span class="line"> rehash();</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>这里其实比较重要的就是前面的 Entry 的构造方法，Entry 是个 WeakReference 的子类，然后在构造方法里可以看到 key 会被包装成一个弱引用，这里为什么使用弱引用，其实是方便这个 key 被回收，如果前面的 ThreadLocal tl实例被设置成 null 了，如果这里是直接的强引用的话，就只能等到线程整个回收了，但是其实是弱引用也会有问题，主要是因为这个 value，如果在 ThreadLocal tl 被设置成 null 了，那么其实这个 value 就会没法被访问到，所以最好的操作还是在使用完了就 remove 掉</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>ThreadLocal</tag>
|
|
<tag>弱引用</tag>
|
|
<tag>内存泄漏</tag>
|
|
<tag>WeakReference</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊厦门旅游的好与不好</title>
|
|
<url>/2021/04/11/%E8%81%8A%E8%81%8A%E5%8E%A6%E9%97%A8%E6%97%85%E6%B8%B8%E7%9A%84%E5%A5%BD%E4%B8%8E%E4%B8%8D%E5%A5%BD/</url>
|
|
<content><![CDATA[<p>这几天去了趟厦门，原来几年前就想去了，本来都请好假了，后面因为一些事情没去成，这次刚好公司组织，就跟 LD 一起去了厦门，也不洋洋洒洒地写游记了，后面可能会有，今天先来总结下好的地方和比较坑的地方。<br>这次主要去了中山路、鼓浪屿、曾厝(cuo)垵、植物园、灵玲马戏团，因为住的离环岛路比较近，还有幸现场看了下厦门马拉松，其中</p>
|
|
<h2 id="中山路"><a href="#中山路" class="headerlink" title="中山路"></a>中山路</h2><p>这里看上去是有点民国时期的建筑风格，部分像那种电视里的租界啥的，不过这次去的时候都在翻修，路一大半拦起来了，听导游说这里往里面走有个局口街，然后上次听前同事说厦门比较有名的就是沙茶面和海蛎煎，不出意料的不太爱吃，沙茶面比较普通，可能是没吃到正宗的，海蛎煎吃不惯，倒是有个大叔的沙茶里脊还不错，在局口街那，还有小哥在那拍，应该也算是个网红打卡点了，然后吃了个油条麻糍也还不错，总体如果是看建筑的话可能最近不是个好时间，个人也没这方面爱好，吃的话最好多打听打听沙茶面跟海蛎煎哪里正宗。如果不知道哪家好吃，也不爱看这类建筑的可以排个坑。</p>
|
|
<h2 id="鼓浪屿"><a href="#鼓浪屿" class="headerlink" title="鼓浪屿"></a>鼓浪屿</h2><p>鼓浪屿也是完全没啥概念，需要乘船过去，但是只要二十分钟，岛上没有机动车，基本都靠走，有几个比较有名的地方，菽庄花园，里面有钢琴博物馆，对这个感兴趣的可以去看看，旁边是沙滩还可以逛逛，然后有各种博物馆，风琴啥的，岛上最大的特色是巷子多，道听途说有三百多条小巷，还有几个网红打卡点，周杰伦晴天墙，还有个最美转角，都是挤满了人排队打卡拍照，不过如果不着急，慢慢悠悠逛逛还是不错的，比较推荐，推荐值☆☆</p>
|
|
<h2 id="曾厝垵"><a href="#曾厝垵" class="headerlink" title="曾厝垵"></a>曾厝垵</h2><p>一直读不对这个字，都是叫：那个曾什么垵，愧对语文老师，这里到算是意外之喜，鼓浪屿回来已经挺累了，不过由于比较饿（什么原因后面说），并且离住的地方不远，就过去逛了逛，东西还蛮好吃的，芒果挺便宜，一大杯才十块，无骨鸡爪很贵，不是特别爱，臭豆腐不错的，也不算很贵，这里想起来，那边八婆婆的豆乳烧仙草还不错的，去中山路那会喝了，来曾厝垵也买了，奶茶爱好者可以试试，含糖量应该很高，不爱甜食或者减肥的同学慎重考虑好了再尝试，晚上那边从牌坊出来，沿着环岛路挺多夜宵店什么的，非常推荐，推荐值☆☆☆☆</p>
|
|
<h2 id="植物园"><a href="#植物园" class="headerlink" title="植物园"></a>植物园</h2><p>植物园还是挺名副其实的，有热带植物，沙漠多肉，因为赶时间逛得不多，热带雨林植物那太多人了，都是在那拍照，而且我指的拍照都是拍人照，本身就很小的路，各种十八线网红，或者普通游客在那摆 pose 拍照，挺无语的；沙漠多肉比较惊喜，好多比人高的仙人掌，一大片的仙人球，很可恶的是好多大仙人掌上都有人刻字，越来越体会到，我们社会人多了，什么样的都有，而且不少；还看了下百花厅，但没什么特别的，可能赶时间比较着急，没仔细看，比较推荐，推荐值☆☆☆</p>
|
|
<h2 id="灵玲马戏团"><a href="#灵玲马戏团" class="headerlink" title="灵玲马戏团"></a>灵玲马戏团</h2><p>对这个其实比较排斥，主要是比较晚了，跑的有点远（我太懒了），一开始真的挺拉低体验感受的，上来个什么书法家，现场画马，卖画；不过后面的还算值回票价，主题是花木兰，空中动作应该很考验基本功，然后那些老外的飞轮还跳绳（不知道学名叫啥），动物那块不太忍心看，应该是吃了不少苦头，不过人都这样就往后点再心疼动物吧。</p>
|
|
<h2 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h2><p>厦门是个非常适合干饭人的地方，吃饭的地方大部分是差不多一桌菜十个左右就完了，而且上来就一大碗饭，一瓶雪碧一瓶可乐，对于经常是家里跟亲戚吃饭都得十几二十个菜的乡下人来说，不太吃得惯这样的🤦♂️，当然很有可能是我们预算不足，点的差。但是有一点是我回杭州深有感触的，感觉杭州司机的素质真的是跟厦门的司机差了比较多，杭州除非公交车停了，否则人行道很难看到主动让人的，当然这里拿厦门这个旅游城市来对比也不是很公平，不过这也是体现城市现代化文明水平的一个维度吧。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>旅游</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>杭州</tag>
|
|
<tag>旅游</tag>
|
|
<tag>厦门</tag>
|
|
<tag>中山路</tag>
|
|
<tag>局口街</tag>
|
|
<tag>鼓浪屿</tag>
|
|
<tag>曾厝垵</tag>
|
|
<tag>植物园</tag>
|
|
<tag>马戏团</tag>
|
|
<tag>沙茶面</tag>
|
|
<tag>海蛎煎</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊如何识别和意识到日常生活中的各类危险</title>
|
|
<url>/2021/06/06/%E8%81%8A%E8%81%8A%E5%A6%82%E4%BD%95%E8%AF%86%E5%88%AB%E5%92%8C%E6%84%8F%E8%AF%86%E5%88%B0%E6%97%A5%E5%B8%B8%E7%94%9F%E6%B4%BB%E4%B8%AD%E7%9A%84%E5%90%84%E7%B1%BB%E5%8D%B1%E9%99%A9/</url>
|
|
<content><![CDATA[<p>这篇博客的灵感又是来自于我从绍兴来杭州的路上，在我们进站以后上电梯快到的时候，突然前面不动了，右边我能看到的是有个人的行李箱一时拎不起来，另一边后面看到其实是个小孩子在那哭闹，一位妈妈就在那停着安抚或者可能有点手足无措，其实这一点应该是在几年前慢慢意识到是个非常危险的场景，特别是像绍兴北站这样上去站台是非常长的电梯，因为最近扩建改造，车次减少了很多，所以每一班都有很多人，检票上站台的电梯都是满员运转，试想这种情况，如果刚才那位妈妈再多停留一点时间，很可能就会出现后面的人上不来被挤下去，再严重点就是踩踏事件，但是这类情况很少人真的意识到，非常明显的例子就是很多人拿着比较大比较重的行李箱，不走垂梯，并且在快到的时候没有提前准备好，有可能在玩手机啥的，如果提不动，后面又是挤满人了，就很可能出现前面说的这种情况，并且其实这种是非紧急情况，大多数人都没有心理准备，一旦发生后果可能就会很严重，例如火灾地震疏散大部分人或者说负责引导的都是指示要有序撤离，防止踩踏，但是普通坐个扶梯，一般都不会有这个意识，但是如果这个时间比较长，出现了人员站不住往后倒了，真的会很严重。所以如果自己是带娃的或者带了很重的行李箱的，请提前做好准备，看到前面有人带的，最好也保持一定距离。<br>还有比如日常走路，旁边有车子停着的情况，比较基本的看车灯有没有亮着，亮着的是否是倒车灯，这种应该特别注意远离，至少保持距离，不能挨着走，很多人特别是一些老年人，在一些人比较多的路上，往往完全无视旁边这些车的状态，我走我的路，谁敢阻拦我，管他车在那动不动，其实真的非常危险，车子本身有视线死角，再加上司机的驾驶习惯和状态，想去送死跟碰瓷的除外，还有就是有一些车会比较特殊，车子发动着，但是没灯，可能是车子灯坏了或者司机通过什么方式关了灯，这种比较难避开，不过如果车子打着了，一般会有比较大的热量散发，车子刚灭了也会有，反正能远离点尽量远离，从轿车的车前面走过挨着走要比从屁股后面挨着走稍微安全一些，但也最好不要挨着车走。<br>最后一点其实是我觉得是我自己比较怕死，一般对来向的车或者从侧面出来的车会做更长的预判距离，特别是电瓶车，一般是不让人的，像送外卖的小哥，的确他们不太容易，但是真的很危险啊，基本就生死看刹车，能刹住就赚了，刹不住就看身子骨扛不扛撞了，只是这里要多说点又要谈到资本的趋利性了，总是想法设法的压榨以获取更多的利益，也不扯远了，能远离就远离吧。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>糟心事</tag>
|
|
<tag>扶梯</tag>
|
|
<tag>踩踏</tag>
|
|
<tag>安全</tag>
|
|
<tag>电瓶车</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊对 FunctionalInterface 注解的一些理解</title>
|
|
<url>/2023/10/15/%E8%81%8A%E8%81%8A%E5%AF%B9-FunctionalInterface-%E6%B3%A8%E8%A7%A3%E7%9A%84%E4%B8%80%E4%BA%9B%E7%90%86%E8%A7%A3/</url>
|
|
<content><![CDATA[<p>在看 Tomcat 代码过程中碰到了一个小问题，可以用代码来举例讲一下<br>先定义一个简单的接口</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@FunctionalInterface</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">interface</span> <span class="title class_">FunctionalInterfaceTest</span> {</span><br><span class="line"></span><br><span class="line"> String <span class="title function_">getInfo</span><span class="params">()</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>functionalInterface本质还是个接口，所以可以用类实现接口的方式<br>比如像这样</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title class_">FunctionalInterfaceImpl</span> <span class="keyword">implements</span> <span class="title class_">FunctionalInterfaceTest</span>{</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="keyword">public</span> String <span class="title function_">getInfo</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="string">"info"</span>;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>但是有下面另外两种方式也是可以实现接口的<br>第一种是 lambda，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">FunctionalInterfaceTest</span> <span class="variable">interfaceTest</span> <span class="operator">=</span> () -> <span class="string">"aaa"</span>;</span><br><span class="line">System.out.println(interfaceTest.getInfo());</span><br></pre></td></tr></table></figure>
|
|
<p>这个实现方式就是直接把 lambda 表达式作为接口实现，但是如果这样就要注意如果实在代码中间的话，debug 还是会跳到这个 lambda 的实现，有时候会混乱，因为只是跳转到 lambda 表达式，而不是所在的方法<br>第二种是这样</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> <span class="type">FunctionalInterfaceTest</span> <span class="variable">interfaceTest1</span> <span class="operator">=</span> demo.getSelfInfo();</span><br><span class="line"> System.out.println(interfaceTest.getInfo());</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> FunctionalInterfaceTest <span class="title function_">getSelfInfo</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">this</span>::info;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> String <span class="title function_">info</span><span class="params">()</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="string">"ccc"</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>前面 <code>getSelfInfo</code> 的返回值就是类里的一个方法，可以用来作为 FunctionalInterface 的实现，这里跟上面有个比较奇怪的，或者说对于常规的接口理解，一般来说实现的方法签名必须和接口的方法签名一致，但是对于函数接口，私域就突破了这个限制，不管是lambda 这种匿名函数或者后面这种方法引用，都没有用 <code>getInfo</code> 这个方法名，这也是一个比较最近的一个理解。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
<tag>stream</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊我刚学会的应用诊断方法</title>
|
|
<url>/2020/05/22/%E8%81%8A%E8%81%8A%E6%88%91%E5%88%9A%E5%AD%A6%E4%BC%9A%E7%9A%84%E5%BA%94%E7%94%A8%E8%AF%8A%E6%96%AD%E6%96%B9%E6%B3%95/</url>
|
|
<content><![CDATA[<p>因为传说中的出身问题，我以前写的是PHP，在使用 swoole 之前，基本的应用调试手段就是简单粗暴的 var_dump，exit，对于单进程模型的 PHP 也是简单有效，技术栈换成 Java 之后，就变得没那么容易，一方面是需要编译，另一方面是一般都是基于 spring 的项目，如果问题定位比较模糊，那框架层的是很难靠简单的 System.out.println 或者打 log 解决，(PS：我觉得可能我写的东西比较适合从 PHP 这种弱类型语言转到 Java 的小白同学)这个时候一方面因为是 Java，有了非常好用的 idea IDE 的支持，可以各种花式调试，条件断点尤其牛叉，但是又因为有 Spring+Java 的双重原因，有些情况下单步调试可以把手按废掉，这也是我之前一直比较困惑苦逼的点，后来随着慢慢精（jiang）进（you）之后，比如对于一个 oom 的情况，我们可以通过启动参数加上-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=xx/xx 来配置溢出时的堆dump 日志，获取到这个文件后，我们可以通过像 Memory Analyzer (MAT)[<a href="https://www.eclipse.org/mat/]">https://www.eclipse.org/mat/]</a> (The Eclipse Memory Analyzer is a fast and feature-rich Java heap analyzer that helps you find memory leaks and reduce memory consumption.)来查看诊断问题所在，之前用到的时候是因为有个死循环一直往链表里塞数据，属于比较简单的，后来一次是由于运维进行应用迁移时按默认的统一配置了堆内存大小，导致内存的确不够用，所以溢出了，<br>今天想说的其实主要是我们的 thread dump，这也是我最近才真正用的一个方法，可能真的很小白了，用过 ide 的单步调试其实都知道会有一个一层层的玩意，比如函数从 A，调用了 B，再从 B 调用了 C，一直往下(因为是 Java，所以还有很多🤦♂️)，这个其实也是大部分语言的调用模型，利用了栈这个数据结构，通过这个结构我们可以知道代码的调用链路，由于对于一个 spring 应用，在本身框架代码量非常庞大的情况下，外加如果应用代码也是非常多的时候，有时候通过单步调试真的很难短时间定位到问题，需要非常大的耐心和仔细观察，当然不是说完全不行，举个例子当我的应用经常启动需要非常长的时间，因为本身应用有非常多个 bean，比较难说究竟是 bean 的加载的确很慢还是有什么异常原因，这种时候就可以使用 thread dump 了，具体怎么操作呢<br><img data-src="https://mystore-1255223546.cos.ap-chengdu.myqcloud.com/uPic/GPYHjd.png"><br>如果在idea 中运行或者调试时，可以直接点击这个照相机一样的按钮，右边就会出现了左边会显示所有的线程，右边会显示线程栈，</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">"main@1" prio=5 tid=0x1 nid=NA runnable</span><br><span class="line"> java.lang.Thread.State: RUNNABLE</span><br><span class="line"> at TreeDistance.treeDist(TreeDistance.java:64)</span><br><span class="line"> at TreeDistance.treeDist(TreeDistance.java:65)</span><br><span class="line"> at TreeDistance.treeDist(TreeDistance.java:65)</span><br><span class="line"> at TreeDistance.treeDist(TreeDistance.java:65)</span><br><span class="line"> at TreeDistance.main(TreeDistance.java:45)</span><br></pre></td></tr></table></figure>
|
|
<p>这就是我们主线程的堆栈信息了，main 表示这个线程名，prio表示优先级，默认是 5，tid 表示线程 id，nid 表示对应的系统线程，后面的runnable 表示目前线程状态，因为是被我打了断点，所以是就许状态，然后下面就是对应的线程栈内容了，在<code>TreeDistance</code>类的 <code>treeDist</code>方法中，对应的文件行数是 64 行。<br>这里使用 thread dump一般也不会是上面我截图代码里的这种代码量很少的，一般是大型项目，有时候跑着跑着没反应，又不知道跑到哪了，特别是一些刚接触的大项目或者需要定位一个大项目的一个疑难问题，一时没思路时，可以使用这个方法，个人觉得非常有帮助。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>工具</category>
|
|
<category>Thread dump</category>
|
|
<category>问题排查</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Thread dump</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊我理解的分布式事务</title>
|
|
<url>/2020/05/17/%E8%81%8A%E8%81%8A%E6%88%91%E7%90%86%E8%A7%A3%E7%9A%84%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/</url>
|
|
<content><![CDATA[<p>前面说了mysql数据库的事务相关的，那事务是用来干嘛的，这里得补一下ACID，</p>
|
|
<blockquote>
|
|
<p><a href="https://zh.wikipedia.org/wiki/ACID"><strong>ACID</strong></a>，是指<a href="https://zh.wikipedia.org/wiki/%E6%95%B0%E6%8D%AE%E5%BA%93%E7%AE%A1%E7%90%86%E7%B3%BB%E7%BB%9F">数据库管理系统</a>（<a href="https://zh.wikipedia.org/wiki/DBMS">DBMS</a>）在写入或更新资料的过程中，为保证<a href="https://zh.wikipedia.org/wiki/%E6%95%B0%E6%8D%AE%E5%BA%93%E4%BA%8B%E5%8A%A1">事务</a>（transaction）是正确可靠的，所必须具备的四个特性：<a href="https://zh.wikipedia.org/w/index.php?title=%E5%8E%9F%E5%AD%90%E6%80%A7&action=edit&redlink=1">原子性</a>（atomicity，或称不可分割性）、<a href="https://zh.wikipedia.org/w/index.php?title=%E4%B8%80%E8%87%B4%E6%80%A7_(%E6%95%B0%E6%8D%AE%E5%BA%93%E7%B3%BB%E7%BB%9F)&action=edit&redlink=1">一致性</a>（consistency）、<a href="https://zh.wikipedia.org/wiki/%E9%9A%94%E9%9B%A2%E6%80%A7">隔离性</a>（isolation，又称独立性）、<a href="https://zh.wikipedia.org/w/index.php?title=%E6%8C%81%E4%B9%85%E6%80%A7&action=edit&redlink=1">持久性</a>（durability）。</p>
|
|
</blockquote>
|
|
<ul>
|
|
<li><p>Atomicity（原子性）：一个事务（transaction）中的所有操作，或者全部完成，或者全部不完成，不会结束在中间某个环节。事务在执行过程中发生错误，会被<a href="https://zh.wikipedia.org/wiki/%E5%9B%9E%E6%BB%9A_(%E6%95%B0%E6%8D%AE%E7%AE%A1%E7%90%86)">回滚</a>（Rollback）到事务开始前的状态，就像这个事务从来没有执行过一样。即，事务不可分割、不可约简。[<a href="https://zh.wikipedia.org/wiki/ACID#cite_note-acid-1">1]</a></p>
|
|
</li>
|
|
<li><p>Consistency（一致性）：在事务开始之前和事务结束以后，数据库的完整性没有被破坏。这表示写入的资料必须完全符合所有的预设<a href="https://zh.wikipedia.org/wiki/%E6%95%B0%E6%8D%AE%E5%AE%8C%E6%95%B4%E6%80%A7">约束</a>、<a href="https://zh.wikipedia.org/wiki/%E8%A7%A6%E5%8F%91%E5%99%A8_(%E6%95%B0%E6%8D%AE%E5%BA%93)">触发器</a>、<a href="https://zh.wikipedia.org/wiki/%E7%BA%A7%E8%81%94%E5%9B%9E%E6%BB%9A">级联回滚</a>等。[<a href="https://zh.wikipedia.org/wiki/ACID#cite_note-acid-1">1]</a></p>
|
|
</li>
|
|
<li><p>Isolation（隔离性）：数据库允许多个并发事务同时对其数据进行读写和修改的能力，隔离性可以防止多个事务并发执行时由于交叉执行而导致数据的不一致。事务隔离分为不同级别，包括未提交读（Read uncommitted）、提交读（read committed）、可重复读（repeatable read）和串行化（Serializable）。[<a href="https://zh.wikipedia.org/wiki/ACID#cite_note-acid-1">1]</a></p>
|
|
</li>
|
|
<li><p>Durability（持久性）：事务处理结束后，对数据的修改就是永久的，即便系统故障也不会丢失。[<a href="https://zh.wikipedia.org/wiki/ACID#cite_note-acid-1">1]</a></p>
|
|
</li>
|
|
</ul>
|
|
<p>在mysql中，借助于MVCC，各种级别的锁，日志等特性来实现了事务的ACID，但是这个我们通常是对于一个数据库服务的定义，常见的情况下我们的数据库随着业务发展也会从单实例变成多实例，组成主从Master-Slave架构，这个时候其实会有一些问题随之出现，比如说主从同步延迟，假如在业务代码中做了读写分离，对于一些敏感度较低的数据其实问题不是很大，只要主从延迟不到特别夸张的地步一般都是可以忍受的，但是对于一些核心的业务数据，比如订单之类的，不能忍受数据不一致，下了单了，付了款了，一刷订单列表，发现这个订单还没支付，甚至订单都没在，这对于用户来讲是恨不能容忍的错误，那么这里就需要一些措施，要不就不读写分离，要不就在 redis 这类缓存下订单，或者支付后加个延时等，这些都是一些补偿措施，并且这也是一个不太切当的例子，比较合适的例子也可以用这个下单来说，一般在电商平台下单会有挺多要做的事情，比如像下面这个图</p>
|
|
<p><img data-src="https://mystore-1255223546.cos.ap-chengdu.myqcloud.com/uPic/WX20200517-2127322x.png"></p>
|
|
<p>下单的是后要冻结核销优惠券，如果账户里有钱要冻结扣除账户里的钱，如果使用了J 豆也一样，可能还有 E 卡，忽略我借用的平台，因为目前一般后台服务化之后，可能每一项都是对应的一个后台服务，我们期望的执行过程是要不全成功，要不就全保持执行前状态，不能是部分扣减核销成功了，部分还不行，所以我们处理这种情况会引入一些通用的方案，第一种叫<a href="https://zh.wikipedia.org/wiki/%E4%BA%8C%E9%98%B6%E6%AE%B5%E6%8F%90%E4%BA%A4">二阶段提交</a>,</p>
|
|
<blockquote>
|
|
<p><strong>二阶段提交</strong>（英语：Two-phase Commit）是指在<a href="https://zh.wikipedia.org/wiki/%E8%AE%A1%E7%AE%97%E6%9C%BA%E7%BD%91%E7%BB%9C">计算机网络</a>以及<a href="https://zh.wikipedia.org/wiki/%E6%95%B0%E6%8D%AE%E5%BA%93">数据库</a>领域内，为了使基于<a href="https://zh.wikipedia.org/wiki/%E5%88%86%E5%B8%83%E5%BC%8F%E7%B3%BB%E7%BB%9F">分布式系统</a>架构下的所有节点在进行<a href="https://zh.wikipedia.org/wiki/%E6%95%B0%E6%8D%AE%E5%BA%93%E4%BA%8B%E5%8A%A1">事务</a>提交时保持一致性而设计的一种<a href="https://zh.wikipedia.org/wiki/%E6%BC%94%E7%AE%97%E6%B3%95">算法</a>。通常，<strong>二阶段提交</strong>也被称为是一种<strong>协议</strong>（Protocol）。在分布式系统中，每个节点虽然可以知晓自己的操作时成功或者失败，却无法知道其他节点的操作的成功或失败。当一个事务跨越多个节点时，为了保持事务的<a href="https://zh.wikipedia.org/wiki/ACID">ACID</a>特性，需要引入一个作为<strong>协调者</strong>的组件来统一掌控所有节点（称作<strong>参与者</strong>）的操作结果并最终指示这些节点是否要把操作结果进行真正的提交（比如将更新后的数据写入磁盘等等）。因此，二阶段提交的算法思路可以概括为： 参与者将操作成败通知协调者，再由协调者根据所有参与者的反馈情报决定各参与者是否要提交操作还是中止操作。</p>
|
|
</blockquote>
|
|
<p>对于上面的例子，我们将整个过程分成两个阶段，首先是提交请求阶段，这个阶段大概需要做的是确定资源存在，锁定资源，可能还要做好失败后回滚的准备，如果这些都 ok 了那么就响应成功，这里其实用到了一个叫事务的协调者的角色，类似于裁判员，每个节点都反馈第一阶段成功后，开始执行第二阶段，也就是实际执行操作，这里也是需要所有节点都反馈成功后才是执行成功，要不就是失败回滚。其实常用的分布式事务的解决方案主要也是基于此方案的改进，比如后面介绍的<a href="https://zh.wikipedia.org/wiki/%E4%B8%89%E9%98%B6%E6%AE%B5%E6%8F%90%E4%BA%A4">三阶段提交</a>，有三阶段提交就是因为二阶段提交比较尴尬的几个点，</p>
|
|
<ul>
|
|
<li>第一是对于两阶段提交，其中默认只有协调者有超时时间，当一个参与者进入卡死状态时只能依赖协调者的超时来结束任务，这中间的时间参与者都是锁定着资源</li>
|
|
<li>第二是协调者的单点问题，万一挂了，参与者就会在那傻等着</li>
|
|
</ul>
|
|
<p>所以三阶段提交引入了各节点的超时机制和一个准备阶段，首先是一个<code>can commit</code>阶段，询问下各个节点有没有资源，能不能进行操作，这个阶段不阻塞，只是提前做个摸底，这个阶段其实人畜无害，但是能提高成功率，在这个阶段如果就有节点反馈是不接受的，那就不用执行下去了，也没有锁资源，然后第二阶段是 <code>pre commit</code> ，这个阶段做的事情跟原来的 第一阶段比较类似，然后是第三阶段<code>do commit</code>，其实三阶段提交我个人觉得只是加了个超时，和准备阶段，好像木有根本性的解决的两阶段提交的问题，后续可以再看看一些论文来思考讨论下。</p>
|
|
<p>2020年05月24日22:11 更新<br>这里跟朋友讨论了下，好像想通了最核心的一点，对于前面说的那个场景，如果是两阶段提交，如果各个节点中有一个没回应，并且协调者也挂了，这个时候会有什么情况呢，再加一个假设，其实比如这个一阶段其实是检验就失败的，理论上应该大家都释放资源，那么对于这种异常情况，其他的参与者就不知所措了，就傻傻地锁着资源阻塞着，那么三阶段提交的意义就出现了，把第一阶段拆开，那么即使在这个阶段出现上述的异常，即也不会锁定资源，同时参与者也有超时机制，在第二阶段锁定资源出现异常是，其他参与者节点等超时后就自动释放资源了，也就没啥问题了，不过对于那种异常恢复后的一些情况还是没有很好地解决，需要借助 zk 等，后面有空可以讲讲 paxos 跟 raft 等</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>分布式事务</category>
|
|
<category>两阶段提交</category>
|
|
<category>三阶段提交</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>分布式事务</tag>
|
|
<tag>两阶段提交</tag>
|
|
<tag>三阶段提交</tag>
|
|
<tag>2PC</tag>
|
|
<tag>3PC</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊下apollo配置中心的接入</title>
|
|
<url>/2024/08/25/%E8%81%8A%E4%B8%8Bapollo%E9%85%8D%E7%BD%AE%E4%B8%AD%E5%BF%83%E7%9A%84%E6%8E%A5%E5%85%A5/</url>
|
|
<content><![CDATA[<p>很多技术栈在优化过程中都会有更便捷的接入方式，或者接入demo，这次想拿apollo来对比一些例如向量数据库的部署方式，对我说的就是milvus，<br>apollo如果生产环境部署完全不推荐用这种方式，但是如果为了做个实验，研究下源码还是很方便的，当然前提是有docker环境<br>首先下载docker-compose配置文件，如果是x86环境就是这个<a href="https://github.com/apolloconfig/apollo-quick-start/blob/master/docker-compose.yml">链接</a>，如果是m1这类的就是用这个<a href="https://github.com/apolloconfig/apollo-quick-start/blob/master/docker-compose-arm64.yml">链接</a>，然后再下载<a href="https://github.com/apolloconfig/apollo-quick-start/tree/master/sql">sql文件夹</a><br>目录结构差不多是这样<br><img data-src="https://img.nicksxs.me/blog/6PvThg.png"><br>然后在这个目录下执行<br><code>sudo docker-compose -f docker-compose-arm64.yml up -d</code><br>如果是非m1的话直接用<br><code>sudo docker-compose up -d</code><br>就好，因为docker-compose默认识别的文件名就是<code>docker-compose.yml</code><br>然后看下日志</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">==== starting service ====</span><br><span class="line">Service logging file is ./service/apollo-service.log</span><br><span class="line">Application is running as root (UID 0). This is considered insecure.</span><br><span class="line">Started [66]</span><br><span class="line">Waiting for config service startup...</span><br><span class="line">Config service started. You may visit http://localhost:8080 for service status now!</span><br><span class="line">Waiting for admin service startup.</span><br><span class="line">Admin service started</span><br><span class="line">==== starting portal ====</span><br><span class="line">Portal logging file is ./portal/apollo-portal.log</span><br><span class="line">Application is running as root (UID 0). This is considered insecure.</span><br><span class="line">Started [211]</span><br><span class="line">Waiting for portal startup...</span><br><span class="line">Portal started. You can visit http://localhost:8070 now!</span><br></pre></td></tr></table></figure>
|
|
<p>就表示启动成功了，然后就可以访问后面那个地址 <code>http://localhost:8070</code> 进入控制台，默认用户名apollo，密码是admin<br>然后我们在应用中想要使用，主要是这个几个配置<br>第一个就是在resources目录下创建<code>apollo-env.properties</code><br>里面是meta server的地址，比如我这边就是</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">dev.meta=http://127.0.0.1:8080</span><br></pre></td></tr></table></figure>
|
|
<p>这表示是对应的<code>spring.profiles.active</code>是<code>dev</code>的配置<br>第二步是添加pom依赖</p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>com.ctrip.framework.apollo<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>apollo-client<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>2.0.1<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"><span class="tag"></<span class="name">dependency</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>最后一步是在springboot的启动类添加注解</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@EnableApolloConfig</span></span><br></pre></td></tr></table></figure>
|
|
<p>然后就可以直接使用<code>@Value</code>注解使用配置的值<br><img data-src="https://img.nicksxs.me/blog/YGxFk2.png"><br>请求下这个接口,就可以看到对应的值<br><img data-src="https://img.nicksxs.me/blog/GHDpvr.png"><br>然后我们可以在控制台修改下这个值，发布<br><img data-src="https://img.nicksxs.me/blog/CJf69T.png"><br>默认日志也会把这个打印出来</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">2024-08-24 19:38:30.462 INFO 57920 --- [Apollo-Config-1] c.f.a.s.p.AutoUpdateConfigChangeListener : Auto update apollo changed value successfully, new value: apolloDemoValue, key: demo, beanName: demoController, field: com.nicksxs.spbdemo.controller.DemoController.demo</span><br></pre></td></tr></table></figure>
|
|
<p>对应的请求也会拿到最新的值<br><img data-src="https://img.nicksxs.me/blog/ZCtRDh.png"></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Apollo</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊我的远程工作体验</title>
|
|
<url>/2022/06/26/%E8%81%8A%E8%81%8A%E6%88%91%E7%9A%84%E8%BF%9C%E7%A8%8B%E5%B7%A5%E4%BD%9C%E4%BD%93%E9%AA%8C/</url>
|
|
<content><![CDATA[<p>发生疫情之后，因为正好是春节假期，假期结束的时候还不具备回工作地点办公的条件，所以史无前例地开始了远程办公，以前对于远程办公的概念还停留在国外一些有“格局”的企业会允许员工远程办公，当然对于远程办公这个事情本身我个人也并不是全然支持的态度，其中涉及到很多方面，首先远程办公并不代表就是不用去办公地点上班，可以在家里摸鱼，相对能够得到较高报酬的能够远程办公的企业需要在远程办公期间能够有高效的产出，并且也需要像在公司办公地点一样，能随时被联系到，第二点是薪资福利之外的社保公积金，除非薪资相比非远程办公的企业高出比较多，不然没法 cover 企业额外缴纳的社保公积金，听说有部分企业也会远程办公点给员工上社保，但是毕竟能做到这点的很少，在允许远程办公的企业数量这个本来就不大的基数里，大概率是少之又少了。<br>疫情这个特殊原因开始的远程办公体验也算是开了个之前不太容易开的头，也跟我前面说的第一点有关系，大部分的企业也会担心员工远程办公是否有与在公司办公地点办公一样或者比较接近的办公效率。同时我们在开始远程办公的时候也碰到了因为原先没做过相应准备而导致的许多问题，首先基础设施上就有几个问题，第一个是办公电脑的问题，因为整个公司各个部门的工作性质和内容不同，并不是每个部门都是配笔记本的，或者有些部门并不需要想研发一样带上电脑 on call，所以那么使用台式机或者没有将笔记本带回家的则需要自己准备电脑或者让公司邮寄。第二个是远程网络的问题，像我们公司有研发团队平时也已经准备好了 vpn，但是在这种时候我们没准备好的是 vpn 带宽，毕竟平时只会偶尔有需要连一下 vpn 到公司网络，像这样大量员工都需要连接 vpn 进行工作的话，我们的初步体验就是网络卡的不行，一些远程调试工作没法进行，并且还有一些问题是可能只有我们研发会碰到，比如我们的线上测试服务器网络在办公地点是有网络打通的，但是我们在家就没办法连接，还有就是沟通效率相关，因为这是个全国性的情况，线上会议工具原先都是为特定用户使用，并且视频音频实时传输所需要的带宽质量要求也是比较高的，大规模的远程会议沟通需求让这些做线上会议的服务也算是碰上了类似双十一的大考了，我们是先后使用了 zoom，腾讯会议跟钉钉视频会议，使用体验上来说是 zoom 做得相对比较成熟和稳定，不过后面腾讯会议跟钉钉视频会议也开始赶上来。<br>前面说的这几个点都是得有远程办公经验的公司才会提前做好相应的准备，比如可以做动态网络扩容，能够在需要大量员工连接公司网络的情况下快速响应提升带宽，另一些则是偏软性的，比如如如何在远程办公的条件下控制我们项目进度，如果保证沟通信息是否能像当面沟通那样准确传达，这方面其实我的经验也是边实操边优化的，最开始我们可能为了高效同步消息，会频繁的使用视频会议沟通，这其实并不能解决沟通效率问题，反而打扰了正常的工作，后续我们在特别是做项目过程中就通过相对简单的每日早会和日报机制，将每天的进度与问题风险点进行同步确认，只与相关直接干系人进行视频电话沟通确认，并且要保持一个思维，即远程办公比较适宜的是相对比较成熟的团队，平常工作和合作都已经有默契或者说规则并且能够遵守，在这个前提下，将目光专注于做的事情而不是管到具体的人有没有全天都在高效工作。同样也希望国内的环境能够有更多的远程火种成长起来，让它成为更好的工作方式，WLB！</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>远程办公</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊最近平淡的生活之又聊通勤</title>
|
|
<url>/2021/11/07/%E8%81%8A%E8%81%8A%E6%9C%80%E8%BF%91%E5%B9%B3%E6%B7%A1%E7%9A%84%E7%94%9F%E6%B4%BB/</url>
|
|
<content><![CDATA[<p>一直以来过着特别平淡普通的生活，不过大多数人应该都这样吧，也许有些人可以把平凡的生活过得精彩，最简单的说明就是朋友圈吧，看我一年的盆友圈虽然在发，不过大概 90%的都是发发跑步的打卡，偶尔会有稀稀拉拉的点赞，天天上班，也不喜欢发什么状态，觉得没什么人关注，索性不发。</p>
|
|
<p>只是这么平淡的生活就有一些自己比较心烦纠结的，之前有提到过的交通，最近似乎又发现了一点，就真相总是让人跌破眼镜，以前觉得我可能是胆子比较小，所以会觉得怎么路上这些电瓶都是这么肆无忌惮的往我冲过来，后面慢慢有一种借用电视剧读心神探的概念，安全距离，觉得大部分人跟我一样，骑电瓶车什么的总还是有个安全距离，只是可能这个安全距离对于不同的人不一样，那些骑电瓶车的潜意识里的安全距离是非常短，所以经常会骑车离着你非常近才会刹车，但是这个安全距离理论最近又被推翻了，因为经历过几次电瓶车就是已经跟你有身体接触了，但是没到把人撞倒的程度，似乎这些骑电瓶车的觉得步行的行人在人行道上是空气，蹭一下也无所谓，反正不能挡我的路，总感觉要不是我在前面骑自行车太慢挡着电瓶车，不然他们都能起飞去干掉 F35 解放湾湾了；</p>
|
|
<p>另一个问题应该是说我们交通规则普及的太少，虽然我们没有路权这个名词概念，但是其实是有这个优先级的，包括像杭州是以公交车在人行道礼让行人闻名的，其实这个文明的行为只限于人行道在直行路中间的，大部分在十字路口，右转的公交车很少会让直行人行道的，前提是直行的绿灯的时候，特别是像公交车这样，车身特别长，右转的时候会有比较大的死角，如果是公交车先转，行人或者自行车很容易被卷进去，非常危险的，私家车就更不用说了，反正右转即使人行道上人非常多要转的也是一秒都不等，所以我自己在开车的时候是尽量在右转的时候等人行道上的行人或者骑车的走完，因为总会觉得我是不是有点双标，骑车走路的时候希望开车的能按规则让我，自己开车的时候又想赶紧开走，所以在开车的时候尽量做到让行车和骑车的。</p>
|
|
<p>还有个其实是写着写着想起来的，比如我骑车左转的时候，因为我是左转到对角那就到了，跟那些左转后要再直行的不一样，我们应该在学车的时候也学过，超车要从左边超，但是往往那些骑电瓶车的在左转的时候会从我右边超过来再往左边撇过去，如果留的空间大还好，有些电瓶车就是如果车头超过了就不管他的车屁股，如果我不减速，自行车就被刮倒了，可能的确是别人就不是人，只要不把你撞倒就无所谓，反正为了你自己不被撞倒你肯定会让的。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>糟心事</tag>
|
|
<tag>规则</tag>
|
|
<tag>电瓶车</tag>
|
|
<tag>骑车</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊最近平淡的生活之《花束般的恋爱》观后感</title>
|
|
<url>/2021/12/31/%E8%81%8A%E8%81%8A%E6%9C%80%E8%BF%91%E5%B9%B3%E6%B7%A1%E7%9A%84%E7%94%9F%E6%B4%BB%E4%B9%8B%E3%80%8A%E8%8A%B1%E6%9D%9F%E8%88%AC%E7%9A%84%E6%81%8B%E7%88%B1%E3%80%8B%E8%A7%82%E5%90%8E%E6%84%9F/</url>
|
|
<content><![CDATA[<p>周末在领导的提议下看了豆瓣的年度榜单，本来感觉没啥心情看的，看到主演有有村架纯就觉得可以看一下，颜值即正义嘛，男主小麦跟女主小娟（后面简称小麦跟小娟）是两个在一次非常偶然的没赶上地铁末班车事件中相识，这里得说下日本这种通宵营业的店好像挺不错的，看着也挺正常，国内估计只有酒吧之类的可以。晚上去的地方是有点暗暗的，好像也有点类似酒吧，旁边有类似于 dj 那种，然后同桌的还有除了男女主的另外一对男女，也是因为没赶上地铁末班车的，但也是陌生人，然后小麦突然看到了有个非常有名的电影人，小娟竟然也认识，然后旁边那对完全不认识，还在那吹自己看过很多电影，比如《肖申克的救赎》，于是男女主都特别鄙夷地看着他们，然后他们又去了另一个有点像泡澡的地方席地而坐，他们发现了自己的鞋子都是一样的，然后在女的去上厕所的时候，小麦暗恋的学姐也来了，然后小麦就去跟学姐他们一起坐了，小娟回来后有点不开心就说去朋友家睡，幸好小麦看出来了（他竟然看出来了，本来以为应该是没填过恋爱很木讷的），就追出去，然后就去了小麦家，到了家小娟发现小麦家的书柜上的书简直就跟她自己家的一模一样，小麦还给小娟吹了头发，一起吃烤饭团，看电影，第二天送小娟上了公交，还约好了一起看木乃伊展，然而并没有交换联系方式，但是他们还是约上了一起看了木乃伊展，在餐馆就出现了片头那一幕的来源，因为餐馆他们想一起听歌，就用有线耳机一人一个耳朵听，但是旁边就有个大叔说“你们是不是不爱音乐，左右耳朵是不一样的，只有一起听才是真正的音乐”这样的话，然后的剧情有点跳，因为是指他们一直在这家餐馆吃饭，中间有他们一起出去玩情节穿插着，也是在这他们确立了关系，可以说主体就是体现了他们非常的合拍和默契，就像一些影评说的，这部电影是说如何跟百分百合拍的人分手，然后就是正常的恋爱开始啪啪啪，一直腻在床上，也没去就业说明会，后面也有讲了一点小麦带着小娟去认识他的朋友，也把小娟介绍给了他们认识，这里算是个小伏笔，后面他们分手也有这里的人的一些关系，接下去的剧情说实话我是不太喜欢的，如果一部八分的电影只是说恋爱被现实打败的话，我觉得在我这是不合格的，但是事实也是这样，小麦其实是有家里的资助，所以后面还是按自己的喜好给一些机构画点插画，小娟则要出去工作，因为小娟家庭观念也是要让她出去有正经工作，用脚指头想也能知道肯定不顺利，然后就是暂时在一家蛋糕店工作，小麦就每天去接小娟，日子过得甜甜蜜蜜，后面小娟在自己的努力下考了个什么资格证，去了一家医院还是什么做前台行政，这中间当然就有父母来见面吃饭了，他们在开始恋爱不久就同居合租了，然后小娟父母就是来说要让她有个正经工作，对男的说的话就是人生就是责任这类的话，而小麦爸爸算是个导火索，因为小麦家里是做烟花生意的，他爸让他就做烟花生意，因为要回老家，并且小麦也不想做，所以就拒绝了，然后他爸就说不给他每个月五万的资助，这也导致了小麦需要去找工作，这个过程也是很辛苦，本来想要年前找好工作，然后事与愿违，后面有一次小娟被同事吐槽怎么从来不去团建，于是她就去了（我以为会拒绝），正在团建的时候小麦给她电话，说找到工作了，是一个创业物流公司这种，这里剧情就是我觉得比较俗套的，小麦各种被虐，累成狗，但是就像小娟爸爸说的话，人生就是责任，所以一直在坚持，但是这样也导致了跟小娟的交流也越来越少，他们原来最爱的漫画，爱玩的游戏，也只剩小娟一个人看，一个人玩，而正是这个时候，小娟说她辞掉了工作，去做一个不是太靠谱的漫画改造的密室逃脱，然后这里其实有一点后面争议很大的，就是这个工作其实是前面小麦介绍给小娟的那些朋友中一个的女朋友介绍的，而在有个剧情就是小娟有一次在这个密室逃脱的老板怀里醒过来，是在 KTV 那样的场景里，这就有很多人觉得小娟是不是出轨了，我觉得其实不那么重要，因为这个离职的事情已经让一切矛盾都摆在眼前，小麦其实是接受这种需要承担责任的生活，也想着要跟小娟结婚，但是小娟似乎还是想要过着那样理想的生活，做自己想做的事情，看自己爱看的漫画，也要小麦能像以前那样一直那么默契的有着相同的爱好，这里的触发点其实还有个是那个小麦的朋友（也就是他女朋友介绍小娟那个不靠谱工作的）的葬礼上，小麦在参加完葬礼后有挺多想倾诉的，而小娟只是想睡了，这个让小麦第二天起来都不想理小娟，只是这里我不太理解，难道这点闹情绪都不能接受吗，所谓的合拍也只是毫无限制的情况下的合拍吧，真正的生活怎么可能如此理想呢，即使没有物质生活的压力，也会有其他的各种压力和限制，在这之后其实小麦想说的是小娟是不是没有想跟自己继续在一起的想法了，而小娟觉得都不说话了，还怎么结婚呢，后面其实导演搞了个小 trick，突然放了异常婚礼，但是不是男女主的，我并不觉得这个桥段很好，在婚礼里男女主都觉得自己想要跟对方说分手了，但是当他们去了最开始一直去的餐馆的时候，一个算是一个现实映照的就是他们一直坐的位子被占了，可能也是导演想通过这个来说明他们已经回不去了，在餐馆交谈的时候，小麦其实是说他们结婚吧，并没有想前面婚礼上预设地要分手，但是小娟放弃了，不想结婚，因为不想过那样的生活了，而小麦觉得可能生活就是那样，不可能一直保持刚恋爱时候的那种感觉，生活就是责任，人生就意味着责任。</p>
|
|
<p>我的一些观点也在前面说了，恋爱到婚姻，即使物质没问题，经济没问题，也会有各种各样的问题，需要一起去解决，因为结婚就意味着需要相互扶持，而不是各取所需，可能我的要求比较高，后面男女主在分手后还一起住了一段时间，我原来还在想会不会通过这个方式让他们继续去磨合同步，只是我失望了，最后给个打分可能是 5 到 6 分吧，勉强及格，好的影视剧应该源于生活高于生活，这一部可能还比不上生活。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>看剧</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊最近平淡的生活之看《神探狄仁杰》</title>
|
|
<url>/2021/12/19/%E8%81%8A%E8%81%8A%E6%9C%80%E8%BF%91%E5%B9%B3%E6%B7%A1%E7%9A%84%E7%94%9F%E6%B4%BB%E4%B9%8B%E7%9C%8B%E3%80%8A%E7%A5%9E%E6%8E%A2%E7%8B%84%E4%BB%81%E6%9D%B0%E3%80%8B/</url>
|
|
<content><![CDATA[<p>其实最近看的不止这一部，前面看了《继承者们》，《少年包青天》这些，就一起聊下，其中看《继承者们》算是个人比较喜欢，以前就有这种看剧的习惯，这个跟《一生一世》里任嘉伦说自己看《寻秦记》看了几十遍一样，我看喜欢的剧也基本上会看不止五遍，继承者们是有帅哥美女看，而且印象中剧情也挺甜的，一般情况下最好是已经有点遗忘剧情了，因为我个人觉得看剧分两种，无聊了又心情不太好，可以看些这类轻松又看过的剧，可以不完全专心地看剧，另外有心情专心看的时候，可以看一些需要思考，一些探案类的或者烧脑类。<br>最近看了《神探狄仁杰》，因为跟前面看的《少年包青天》都是这类古装探案剧，正好有些感想，《少年包青天》算是儿时阴影，本来是不太会去看的，正好有一次有机会跟 LD 一起看了会就也觉得比较有意思就看了下去，不得不说，以前的这些剧还是很不错的，包括剧情和演员，第一部一共是 40 集，看的过程中也发现了大概是五个案子，平均八集一个案子，整体节奏还是比较慢的，但是基本每个案子其实都是构思得很巧妙，很久以前看过但是现在基本不太记得剧情了，每个案子在前面几集的时候基本都猜不到犯案逻辑，但是在看了狄仁杰之后，发现两部剧也有比较大的差别，少年包青天相对来说逻辑性会更强一些，个人主观觉得推理的严谨性更高，可能剧本打磨上更将就一下，而狄仁杰因为要提现他的个人强项，不比少年包青天中有公孙策一时瑜亮的情节，狄仁杰中分工明确，李元芳是个武力担当，曾泰是捧哏的，相对来说是狄仁杰在案子里从始至终地推进案情，有些甚至有些玄乎，会有一些跳脱跟不合理，有些像是狄仁杰的奇遇，不过这些想法是私人的观点，并不是想要评孰优孰劣；第二个感受是不知道是不是年代关系，特别是少年包青天，每个案件的大 boss 基本都不是个完全的坏人，甚至都是比较情有可原的可怜人，因为一些特殊原因，而好几个都是包拯身边的人，这一点其实跟狄仁杰里第一个使团惊魂案件比较类似，虎敬晖也是个人物形象比较丰满的角色，不是个标签化的淡薄形象，跟金木兰的感情和反叛行动在最后都说明的缘由，而且也有随着跟狄仁杰一起办案被其影响感化，最终为了救狄仁杰而被金木兰所杀，只是这样金木兰这个角色就会有些偏执和符号化，当然剧本肯定不是能面面俱到，这样的剧本已经比现在很多流量剧的好很多了。还想到了前阵子看的《指环王》中的白袍萨鲁曼在剧中也是个比较单薄的角色，这样的电影彪炳影史也没办法把个个人物都设计得完整有血有肉，或者说这本来也是应该有侧重点，当然其实我也不觉得指环王就是绝对的最好的，因为相对来说故事情节的复杂性等真的不如西游记，只是在 86 版之后的各种乱七八糟的翻牌和乱拍已经让这个真正的王者神话故事有点力不从心，这里边有部西游记后传是个人还比较喜欢的，虽然武打动作比较鬼畜，但是剧情基本是无敌的，在西游的架构上衍生出来这么完整丰富的故事，人物角色也都有各自的出彩点。<br>说回狄仁杰，在这之前也看过徐克拍的几部狄仁杰的电影版，第一部刘德华拍得相对完成度更高，故事性也可圈可点，后面几部就是剧情拉胯，靠特效拉回来一点分，虽说这个也是所谓的狄仁杰宇宙的构思在里面但是现在来看基本是跟西游那些差不多，完全没有整体性可言，打一枪换一个地方，演员也没有延续性，剧情也是前后跳脱，没什么关联跟承上启下，导致质量层次不一，更不用谈什么狄仁杰宇宙了，不过这个事情也是难说，原因很多，现在资本都是更加趋利的，一些需要更长久时间才能有回报的投资是很难获得资本青睐，所以只能将重心投给选择一些流量明星，而本来应该将资源投给剧本打磨的基本就没了，再深入说也没意义了，社会现状就是这样。<br>还有一点感想是，以前的剧里的拍摄环境还是比较惨的，看着一些房子，甚至皇宫都是比较破旧的，地面还是石板这种，想想以前的演员的环境再想想现在的，比如成龙说的，以前他拍剧就是啪摔了，问这条有没有过，过了就直接送医院，而不是现在可能手蹭破点皮就大叫，甚至还有饭圈这些破事。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>看剧</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊最近平淡的生活之看看老剧</title>
|
|
<url>/2021/11/21/%E8%81%8A%E8%81%8A%E6%9C%80%E8%BF%91%E5%B9%B3%E6%B7%A1%E7%9A%84%E7%94%9F%E6%B4%BB%E4%B9%8B%E7%9C%8B%E7%9C%8B%E8%80%81%E5%89%A7/</url>
|
|
<content><![CDATA[<p>最近因为也没什么好看的新剧和综艺所以就看看一些以前看过的老剧，我是个非常念旧的人吧，很多剧都会反反复复地看，一方面之前看过觉得好看的的确是一直记着，还有就是平时工作完了回来就想能放松下，剧情太纠结的，太烧脑的都不喜欢，也就是我常挂在口头的不喜欢看费脑子的剧，跟我不喜欢狼人杀的原因也类似。</p>
|
|
<p>前面其实是看的太阳的后裔，跟 LD 一起看的，之前其实算是看过一点，但是没有看的很完整，并且很多剧情也忘了，只是这个我我可能看得更少一点，因为最开始的时候觉得男主应该是男二，可能对长得这样的男主并且是这样的人设有点失望，感觉不是特别像个特种兵，但是由于本来也比较火，而且 LD 比较喜欢就从这个开始看了，有两个点是比较想说的</p>
|
|
<p>韩剧虽然被吐槽的很多，但是很多剧的质量，情节把控还是优于目前非常多国内剧的，相对来说剧情发展的前后承接不是那么硬凹出来的，而且人设都立得住，这个是非常重要的，很多国内剧怎么说呢，就是当爹的看起来就比儿子没大几岁，三四十岁的人去演一个十岁出头的小姑娘，除非容貌异常，比如刘晓庆这种，不然就会觉得导演在把我们观众当傻子。瞬间就没有想看下去的欲望了。</p>
|
|
<p>再一点就是情节是大众都能接受度比较高的，现在有很多普遍会找一些新奇的视角，比如卖腐，想某某令，两部都叫某某令，这其实是一个点，延伸出去就是跟前面说的一点有点类似，xx 老祖，人看着就二三十，叫 xx 老祖，（喜欢的人轻喷哈）然后名字有一堆，同一个人物一会叫这个名字，一会又叫另一个名字，然后一堆死表情。</p>
|
|
<p>因为今天有个特殊的事情发生，所以简短的写（shui）一篇了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>看剧</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊这次换车牌及其他</title>
|
|
<url>/2022/02/20/%E8%81%8A%E8%81%8A%E8%BF%99%E6%AC%A1%E6%8D%A2%E8%BD%A6%E7%89%8C%E5%8F%8A%E5%85%B6%E4%BB%96/</url>
|
|
<content><![CDATA[<p>去年 8 月份运气比较好，摇到了车牌，本来其实应该很早就开始摇的，前面第一次换工作没注意社保断缴了一个月，也是大意失荆州，后面到了 17 年社保满两年了，好像只摇了一次，还是就没摇过，有点忘了，好像是什么原因导致那次也没摇成功，但是后面暂住证就取消了，需要居住证，居住证又要一年及以上的租房合同，并且那会买车以后也不怎么开，住的地方车位还好，但是公司车位一个月要两三千，甚至还是打车上下班比较实惠，所以也没放在心上，后面摇到房以后，也觉得应该准备起来车子，就开始办了居住证，居住证其实还可以用劳动合同，而且办起来也挺快，大概是三四月份开始摇，到 8 月份的某一天收到短信说摇到了，一开始还挺开心，不过心里抱着也不怎么开，也没怎么大放在心上，不过这里有一点就是我把那个照片直接发出去，上面有着我的身份证号，被 LD 说了一顿，以后也应该小心点，但是后面不知道是哪里看了下，说杭州上牌已经需要国六标准的车了，瞬间感觉是空欢喜了，可是有同事说是可以的，我就又打了官方的电话，结果说可以的，要先转籍，然后再做上牌。</p>
|
|
<p>转籍其实是很方便的，在交警 12123 App 上申请就行了，在转籍以后，需要去实地验车，验车的话，在支付宝-杭州交警生活号里进行预约，找就近的车管所就好，需要准备一些东西，首先是行驶证，机动车登记证书，身份证，居住证，还有车上需要准备的东西是要有三脚架和反光背心，反光背心是最近几个月开始要的，问过之前去验车的只需要三脚架就好了，预约好了的话建议是赶上班时间越早越好，不然过去排队时间要很久，而且人多了以后会很乱，各种插队，而且有很多都是汽车销售，一个销售带着一堆车，我们附近那个进去的小路没一会就堵满车，进去需要先排队，然后扫码，接着交资料，这两个都排着队，如果去晚了就要排很久的队，交完资料才是排队等验车，验车就是打开引擎盖，有人会帮忙拓印发动机车架号，然后验车的会各种检查一下，车里面，还有后备箱，建议车内整理干净点，后备箱不要放杂物，检验完了之后，需要把三脚架跟反光背心放在后备箱盖子上，人在旁边拍个照，然后需要把车牌遮住后再拍个车子的照片，再之后就是去把车牌卸了，这个多吐槽下，那边应该是本来那边师傅帮忙卸车牌，结果他就说是教我们拆，虽然也不算难，但是不排除师傅有在偷懒，完了之后就是把旧车牌交回去，然后需要在手机上（警察叔叔 App）提交各种资料，包括身份证，行驶证，机动车登记证书，提交了之后就等寄车牌过来了。</p>
|
|
<p>这里面缺失的一个环节就是选号了，选号杭州有两个方式，一种就是根据交管局定期发布的选号号段，可以自定义拼 20 个号，在手机上的交警 12123 App 上可以三个一组的形式提交，如果有没被选走的，就可以预选到这个了，但是这种就是也需要有一定策略，最新出的号段能选中的概率大一点，然后数字全是 8，6 这种的肯定会一早就被选走，然后如果跟我一样可以提前选下尾号，因为尾号数字影响限号，我比较有可能周五回家，所以得避开 5，0 的，第二种就是 50 选一跟以前新车选号一样，就不介绍了。第一种选中了以后可以在前面交还旧车牌的时候填上等着寄过来了，因为我是第一种选中的，第二种也可以在手机上选，也在可以在交还车牌的时候现场选。</p>
|
|
<p>总体过程其实是 LD 在各种查资料跟帮我跑来跑去，要不是 LD，估计在交管局那边我就懵逼了，各种插队，而且车子开着车子，也不能随便跑，所以建议办这个的时候有个人一起比较好。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>换车牌</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊给亲戚朋友的老电脑重装系统那些事儿</title>
|
|
<url>/2021/05/09/%E8%81%8A%E8%81%8A%E7%BB%99%E4%BA%B2%E6%88%9A%E6%9C%8B%E5%8F%8B%E7%9A%84%E8%80%81%E7%94%B5%E8%84%91%E9%87%8D%E8%A3%85%E7%B3%BB%E7%BB%9F%E9%82%A3%E4%BA%9B%E4%BA%8B%E5%84%BF/</url>
|
|
<content><![CDATA[<p>前面这个五一回去之前，LD 姐姐跟我说电脑很卡了，想让我重装系统，问了下 LD 可能是那个 09 年买的笔记本，想想有点害怕[捂脸]，前年有一次好像让我帮忙装了她同事的一个三星的笔记本，本着一些系统洁癖，所以就从开始找纯净版的 win7 家庭版，因为之前那些本基本都自带 win7 的家庭版，而且把激活码就贴在机器下面，然后从三星官网去找官方驱动，还好这个机型的驱动还在，先做了系统镜像，其实感觉这种情况需要两个 U 盘，一个 U 盘装系统作为安装启动盘，一个放驱动，毕竟不是专业装系统的，然后因为官方驱动需要一个个下载一个个安装，然后驱动文件下载的地方还没标明是 32 位还是 64 位的，结果还被 LD 姐姐催着，一直问好没好，略尴尬，索性还是找个一键安装的</p>
|
|
<p>这次甚至更夸张，上次还让带回去，我准备好了系统镜像啥的，第二天装，这次直接带了两个老旧笔记本过来说让当天就装好，感觉有点像被当修电脑的使，又说这些电脑其实都不用了的，都是为了她们当医生的要每年看会课，然后只能用电脑浏览器看，结果都在用 360 浏览器，真的是万恶的 360，其实以前对 360 没啥坏印象，毕竟以前也经常用，只是对于这些老电脑，360 全家桶真的就是装了就废了，2G 的内存，开机就开着 360 安全卫士，360 杀毒，有一个还装了腾讯电脑管家，然后腾讯视频跟爱奇艺也开机启动了，然后还打开 360 浏览器看课，就算再好的系统也吃不消这么用，重装了系统，还是这么装这些东西，也是分分钟变卡，可惜他们都没啥这类概念。</p>
|
|
<p>对于他们要看的课，更搞笑的是，明明在页面上注明了说要使用 IE 浏览器，结果他们都在用 360 浏览器看，但是这个也不能完全怪他们，因为实在是现在的 IE 啥的也有开始不兼容 flash 的配置，需要开启兼容配置，但是只要开启了之后就可以直接用 IE 看，比 360 靠谱很多， 资源占用也比较少，360 估计是基于 chromium 加了很多内置的插件，本身 chromium 也是内存大户，但是说这些其实他们也不懂，总觉得找我免费装下系统能撑一段时间，反正对我来说也应该很简单（他们觉得），实际上开始工作以后，我自己想装个双系统都是上淘宝买别人的服务装的，台式机更是几年没动过系统了，因为要重装一大堆软件，数据备份啥的，还有驱动什么的，分区格式，那些驱动精灵啥的也都是越来越坑，一装就给你带一堆垃圾软件。</p>
|
|
<p>感悟是，总觉得学计算机的就应该会装系统，会修电脑，之前亲戚还拿着一个完全开不起来的笔记本让我来修，这真的是，我说可以找官方维修的，结果我说我搞不定，她直接觉得是修不好了，直接电脑都懒得拿回去了，后面又一次反复解释了才明白，另外就是 360 全家桶，别说老电脑了，新机器都不太吃得消。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>装电脑</tag>
|
|
<tag>老电脑</tag>
|
|
<tag>360 全家桶</tag>
|
|
<tag>修电脑的</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊那些加塞狗</title>
|
|
<url>/2021/01/17/%E8%81%8A%E8%81%8A%E9%82%A3%E4%BA%9B%E5%8A%A0%E5%A1%9E%E7%8B%97/</url>
|
|
<content><![CDATA[<p>今天真的是被气得不轻，情况是碰到一个有 70 多秒的直行红灯，然后直行就排了很长的队，但是左转车道没车，就有好几辆车占着左转车道，准备往直行车道插队加塞，一般这种加塞的，会挑个不太计较的，如果前面一辆不让的话就再等等，我因为赶着回家，就不想让，结果那辆车几次车头直接往里冲，当时怒气值基本已经蓄满了，我真的是分毫都不想让，如果路上都是让着这种人的，那么这种情况只会越来越严重，我理解的这种心态，就赌你怕麻烦，多一事不如少一事，结果就是每次都能顺利插队加塞，其实延伸到我们社会中的种种实质性的排队或者等同于排队的情况，都已经有这种惯有思维，一方面这种不符合规则，可能在严重程度上容易被很多人所忽视，基本上已经被很多人当成是“合理”行为，另一方面，对于这些“微小”的违规行为，本身管理层面也基本没有想要管的意思，就更多的成为了纵容这些行为的导火索，并且大多数人都是想着如果不让，发生点小剐小蹭的要浪费很多时间精力来处理，甚至会觉得会被别人觉得自己太小气等等，诸多内外成本结合起来，会真的去硬刚的可能少之又少了，这样也就让更多的人觉得这种行为是被默许的，再举个非常小的例子，以我们公司疫情期间的盒饭发放为例，有两个比较“有意思”的事情，第一个就是因为疫情，本来是让排队要间隔一米，但是可能除了我比较怕死会跟前面的人保持点距离基本没别人会不挨着前面的人，甚至我跟我前面的人保持点距离，后面的同学会推着我让我上去；第二个是关于拿饭，这么多人排着队拿饭，然后有部分同学，一个人拿好几份，帮组里其他人的都拿了，有些甚至一个人拿十份，假如这个盒饭发放是说明了可以按部门直接全领了那就没啥问题，但是当时的状况是个人排队领自己的那一份，如果一个同学直接帮着组里十几个人都拿了，后面排队的人是什么感受呢，甚至有些是看到队伍排长了，就找队伍里自己认识的比较靠前的人说你帮我也拿一份，其实作为我这个比较按规矩办事的“愣头青”来说，我是比较不能接受这两件小事里的行为的，再往下说可能就有点偏激了，先说到这~</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>开车</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>开车</tag>
|
|
<tag>加塞</tag>
|
|
<tag>糟心事</tag>
|
|
<tag>规则</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊部分公交车的设计bug</title>
|
|
<url>/2021/12/05/%E8%81%8A%E8%81%8A%E9%83%A8%E5%88%86%E5%85%AC%E4%BA%A4%E8%BD%A6%E7%9A%84%E8%AE%BE%E8%AE%A1bug/</url>
|
|
<content><![CDATA[<p>今天惯例坐公交回住的地方，不小心撞了头，原因是我们想坐倒数第二排，然后LD 走在我后面，我就走到最后一排中间等着，但是最后一排是高一截的，等 LD 坐进去以后，我就往前走，结果撞到了车顶的扶手杆子的一端，差点撞昏了去，这里我觉得其实杆子长度应该短一点，不然从最后一排出来，还是有比较大概率因为没注意看而撞到头，特别是没注意看的情况，发力其实会比较大，一头撞上就会像我这样，眼前一黑，又痛得要死。<br>还有一点就是座位设计了，先来看个图<br><img data-src="https://img.nicksxs.me/uPic/sCM2t1.png"><br>图里大致画了两条线，因为可能是轮胎还是什么原因，后排中间会有那么大的突起，但是看两条红线可以发现，靠近过道的座位边缘跟地面突起的边缘不是一样宽的，这样导致的结果就是坐着的时候有一个脚没地儿搁，要不就得侧着斜着坐，或者就是一个脚悬空，短程的可能还好，路程远一点还是比较难受的，特别是像我现在这样，大腿外侧有点难受的情况，就会更难受。<br>虽然说这两个点，基本是屁用没有，但是我也是在自己这个博客说说，也当是个树洞了。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>公交</tag>
|
|
<tag>杭州</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>解决 网络文件夹目前是以其他用户名和密码进行映射的 问题</title>
|
|
<url>/2023/04/09/%E8%A7%A3%E5%86%B3-%E7%BD%91%E7%BB%9C%E6%96%87%E4%BB%B6%E5%A4%B9%E7%9B%AE%E5%89%8D%E6%98%AF%E4%BB%A5%E5%85%B6%E4%BB%96%E7%94%A8%E6%88%B7%E5%90%8D%E5%92%8C%E5%AF%86%E7%A0%81%E8%BF%9B%E8%A1%8C%E6%98%A0%E5%B0%84%E7%9A%84/</url>
|
|
<content><![CDATA[<p>之前在使用 smb 协议在 Windows 中共享磁盘使用映射网络驱动器的时候，如果前一次登录过账号密码后面有了改动，或者前一次改错了，<br>就会出现这样的提示<br><img data-src="https://img.nicksxs.me/blog/%E6%8C%87%E5%AE%9A%E7%9A%84%E7%BD%91%E7%BB%9C%E6%96%87%E4%BB%B6%E5%A4%B9.png"><br>应该是 Windows 已经把之前的连接记录下来了，即使是链接不成功的<br>可以通过在 cmd 或者 powershell 执行 <code>net use</code> 命令查看当前已经连接的<br><img data-src="https://img.nicksxs.me/blog/hAKVco.png"><br>这样就可以用命令来把这个删除<br><code>net use [NETNAME] /delete</code><br>比如这边就可以<br><code>net use \\xxxxxxxx\f /delete</code><br>然后再重新输入账号密码就好了<br>关于<code>net use</code>的命令使用方式可以参考</p>
|
|
<figure class="highlight powershell"><table><tr><td class="code"><pre><span class="line">net use [{<<span class="type">DeviceName</span>> | *}] [\\<<span class="type">ComputerName</span>>\<<span class="type">ShareName</span>>[\<<span class="type">volume</span>>]] [{<<span class="type">Password</span>> | *}]] [/<span class="type">user</span>:[<<span class="type">DomainName</span>>\]<<span class="type">UserName</span>] >[/<span class="type">user</span>:[<<span class="type">DottedDomainName</span>>\]<<span class="type">UserName</span>>] [/<span class="type">user</span>: [<<span class="type">UserName</span>@<span class="type">DottedDomainName</span>>] [/<span class="type">savecred</span>] [/<span class="type">smartcard</span>] [{/<span class="type">delete</span> | /<span class="type">persistent</span>:{<span class="type">yes</span> | <span class="type">no</span>}}]</span><br><span class="line"><span class="type">net</span> <span class="type">use</span> [<<span class="type">DeviceName</span>> [/<span class="type">home</span>[{<<span class="type">Password</span>> | *}] [/<span class="type">delete</span>:{<span class="type">yes</span> | <span class="type">no</span>}]]</span><br><span class="line"><span class="type">net</span> <span class="type">use</span> [/<span class="type">persistent</span>:{<span class="type">yes</span> | <span class="type">no</span>}]</span><br></pre></td></tr></table></figure>
|
|
<p><a href="https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/gg651155(v=ws.11)">官方链接</a></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>技巧</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>windows</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>聊聊一次 brew update 引发的血案-202502更新</title>
|
|
<url>/2025/02/02/%E8%81%8A%E8%81%8A%E4%B8%80%E6%AC%A1-brew-update-%E5%BC%95%E5%8F%91%E7%9A%84%E8%A1%80%E6%A1%88-202502%E6%9B%B4%E6%96%B0/</url>
|
|
<content><![CDATA[<p>之前写了这么一篇标题党，只是这个的确是比较头疼的事情，brew更新了下，php就不能用了，这里面主要是 <code>icu4c</code> 这个库的更新导致的，比如最近我又碰到了<br><img data-src="https://img.nicksxs.me/blog/tETXmO.png">, 正好又解决了下<br>因为后续brew在 m系列芯片的mac上有了更新，所以之前那篇需要有一些改动，<br>首先是这个目录 <code>$(brew --prefix)/Homebrew/Library/Taps/homebrew/homebrew-core/Formula</code><br>目前这个替换符 <code>$(brew --prefix)</code> 还是有效的，不过路径变更为了 <code>/opt/homebrew</code>，但是后面的路径改变了，变成了 <code>/Library/Taps/homebrew/homebrew-core/Formula</code><br>中间少了 <code>Homebrew</code> , 这是第一点，<br>第二点是对于这个文件 <code>icu4c.rb</code> 的重新安装<br>因为目前我还在使用 php 7.4 版本，依赖的是 icu4c 的 71 版本，所以就切换到类似于 <code>e3317b86c11c644e88c762e03eb7b310c3337587</code> 这个 commit id 这样，</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">git checkout -b icu4c-71 e3317b86c11c644e88c762e03eb7b310c3337587</span><br></pre></td></tr></table></figure>
|
|
<p>但是由于目前的brew已经没有 switch 命令了，所以只能把最新版本的卸载掉，再进行安装老版本<br>先通过 </p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">brew uninstall icu4c</span><br></pre></td></tr></table></figure>
|
|
<p>卸载老版本的 icu4c<br>然后再使用 </p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">brew reinstall ./icu4c.rb</span><br></pre></td></tr></table></figure>
|
|
<p>安装 71 版本的，这样就能解决这个问题<br>第三个问题会在安装时出现<br>类似于</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">attestation verification failed: Failure while executing; `/usr/bin/env GH_TOKEN=****** /usr/local/bin/gh attestation verify /Users/xxx/Library/Caches/Homebrew/downloads/</span><br></pre></td></tr></table></figure>
|
|
<p>这样的操作，可以通过设置常量的形式来解决</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">export HOMEBREW_NO_VERIFY_ATTESTATIONS=true</span><br></pre></td></tr></table></figure>
|
|
<p>这样就不去做这个验证了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Mac</category>
|
|
<category>PHP</category>
|
|
<category>Homebrew</category>
|
|
<category>PHP</category>
|
|
<category>icu4c</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Mac</tag>
|
|
<tag>PHP</tag>
|
|
<tag>Homebrew</tag>
|
|
<tag>icu4c</tag>
|
|
<tag>zsh</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>记一个容器中 dubbo 注册的小知识点</title>
|
|
<url>/2022/10/09/%E8%AE%B0%E4%B8%80%E4%B8%AA%E5%AE%B9%E5%99%A8%E4%B8%AD-dubbo-%E6%B3%A8%E5%86%8C%E7%9A%84%E5%B0%8F%E7%9F%A5%E8%AF%86%E7%82%B9/</url>
|
|
<content><![CDATA[<p>在目前环境下使用容器部署Java应用还是挺普遍的，但是有一些问题也是随之而来需要解决的，比如容器中应用的dubbo注册，在比较早的版本的dubbo中，就是简单地获取网卡的ip地址。<br>具体代码在这个方法里 <code>com.alibaba.dubbo.config.ServiceConfig#doExportUrlsFor1Protocol</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title function_">doExportUrlsFor1Protocol</span><span class="params">(ProtocolConfig protocolConfig, List<URL> registryURLs)</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">name</span> <span class="operator">=</span> protocolConfig.getName();</span><br><span class="line"> <span class="keyword">if</span> (name == <span class="literal">null</span> || name.length() == <span class="number">0</span>) {</span><br><span class="line"> name = <span class="string">"dubbo"</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">String</span> <span class="variable">host</span> <span class="operator">=</span> protocolConfig.getHost();</span><br><span class="line"> <span class="keyword">if</span> (provider != <span class="literal">null</span> && (host == <span class="literal">null</span> || host.length() == <span class="number">0</span>)) {</span><br><span class="line"> host = provider.getHost();</span><br><span class="line"> }</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">anyhost</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"> <span class="keyword">if</span> (NetUtils.isInvalidLocalHost(host)) {</span><br><span class="line"> anyhost = <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> host = InetAddress.getLocalHost().getHostAddress();</span><br><span class="line"> } <span class="keyword">catch</span> (UnknownHostException e) {</span><br><span class="line"> logger.warn(e.getMessage(), e);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (NetUtils.isInvalidLocalHost(host)) {</span><br><span class="line"> <span class="keyword">if</span> (registryURLs != <span class="literal">null</span> && registryURLs.size() > <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">for</span> (URL registryURL : registryURLs) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">Socket</span> <span class="variable">socket</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Socket</span>();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">SocketAddress</span> <span class="variable">addr</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">InetSocketAddress</span>(registryURL.getHost(), registryURL.getPort());</span><br><span class="line"> socket.connect(addr, <span class="number">1000</span>);</span><br><span class="line"> host = socket.getLocalAddress().getHostAddress();</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> socket.close();</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {}</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> logger.warn(e.getMessage(), e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (NetUtils.isInvalidLocalHost(host)) {</span><br><span class="line"> host = NetUtils.getLocalHost();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>通过jdk自带的方法 <code>java.net.InetAddress#getLocalHost</code>来获取本机地址，这样子对于容器来讲，获取到容器内部ip注册上去其实是没办法被调用到的，<br>而在之后的版本中例如dubbo 2.6.5，则可以通过在docker中设置环境变量的形式来注入docker所在的宿主机地址，<br>代码同样在<code>com.alibaba.dubbo.config.ServiceConfig#doExportUrlsFor1Protocol</code>这个方法中，但是获取host的方法变成了 <code>com.alibaba.dubbo.config.ServiceConfig#findConfigedHosts</code></p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> String <span class="title function_">findConfigedHosts</span><span class="params">(ProtocolConfig protocolConfig, List<URL> registryURLs, Map<String, String> map)</span> {</span><br><span class="line"> <span class="type">boolean</span> <span class="variable">anyhost</span> <span class="operator">=</span> <span class="literal">false</span>;</span><br><span class="line"></span><br><span class="line"> <span class="type">String</span> <span class="variable">hostToBind</span> <span class="operator">=</span> getValueFromConfig(protocolConfig, Constants.DUBBO_IP_TO_BIND);</span><br><span class="line"> <span class="keyword">if</span> (hostToBind != <span class="literal">null</span> && hostToBind.length() > <span class="number">0</span> && isInvalidLocalHost(hostToBind)) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalArgumentException</span>(<span class="string">"Specified invalid bind ip from property:"</span> + Constants.DUBBO_IP_TO_BIND + <span class="string">", value:"</span> + hostToBind);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// if bind ip is not found in environment, keep looking up</span></span><br><span class="line"> <span class="keyword">if</span> (hostToBind == <span class="literal">null</span> || hostToBind.length() == <span class="number">0</span>) {</span><br><span class="line"> hostToBind = protocolConfig.getHost();</span><br><span class="line"> <span class="keyword">if</span> (provider != <span class="literal">null</span> && (hostToBind == <span class="literal">null</span> || hostToBind.length() == <span class="number">0</span>)) {</span><br><span class="line"> hostToBind = provider.getHost();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (isInvalidLocalHost(hostToBind)) {</span><br><span class="line"> anyhost = <span class="literal">true</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> hostToBind = InetAddress.getLocalHost().getHostAddress();</span><br><span class="line"> } <span class="keyword">catch</span> (UnknownHostException e) {</span><br><span class="line"> logger.warn(e.getMessage(), e);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (isInvalidLocalHost(hostToBind)) {</span><br><span class="line"> <span class="keyword">if</span> (registryURLs != <span class="literal">null</span> && !registryURLs.isEmpty()) {</span><br><span class="line"> <span class="keyword">for</span> (URL registryURL : registryURLs) {</span><br><span class="line"> <span class="keyword">if</span> (Constants.MULTICAST.equalsIgnoreCase(registryURL.getParameter(<span class="string">"registry"</span>))) {</span><br><span class="line"> <span class="comment">// skip multicast registry since we cannot connect to it via Socket</span></span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">Socket</span> <span class="variable">socket</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Socket</span>();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="type">SocketAddress</span> <span class="variable">addr</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">InetSocketAddress</span>(registryURL.getHost(), registryURL.getPort());</span><br><span class="line"> socket.connect(addr, <span class="number">1000</span>);</span><br><span class="line"> hostToBind = socket.getLocalAddress().getHostAddress();</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> socket.close();</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable e) {</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> logger.warn(e.getMessage(), e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (isInvalidLocalHost(hostToBind)) {</span><br><span class="line"> hostToBind = getLocalHost();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> map.put(Constants.BIND_IP_KEY, hostToBind);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// registry ip is not used for bind ip by default</span></span><br><span class="line"> <span class="type">String</span> <span class="variable">hostToRegistry</span> <span class="operator">=</span> getValueFromConfig(protocolConfig, Constants.DUBBO_IP_TO_REGISTRY);</span><br><span class="line"> <span class="keyword">if</span> (hostToRegistry != <span class="literal">null</span> && hostToRegistry.length() > <span class="number">0</span> && isInvalidLocalHost(hostToRegistry)) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="title class_">IllegalArgumentException</span>(<span class="string">"Specified invalid registry ip from property:"</span> + Constants.DUBBO_IP_TO_REGISTRY + <span class="string">", value:"</span> + hostToRegistry);</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (hostToRegistry == <span class="literal">null</span> || hostToRegistry.length() == <span class="number">0</span>) {</span><br><span class="line"> <span class="comment">// bind ip is used as registry ip by default</span></span><br><span class="line"> hostToRegistry = hostToBind;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> map.put(Constants.ANYHOST_KEY, String.valueOf(anyhost));</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> hostToRegistry;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p><code>String hostToRegistry = getValueFromConfig(protocolConfig, Constants.DUBBO_IP_TO_REGISTRY);</code><br>就是这一行，</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> String <span class="title function_">getValueFromConfig</span><span class="params">(ProtocolConfig protocolConfig, String key)</span> {</span><br><span class="line"> <span class="type">String</span> <span class="variable">protocolPrefix</span> <span class="operator">=</span> protocolConfig.getName().toUpperCase() + <span class="string">"_"</span>;</span><br><span class="line"> <span class="type">String</span> <span class="variable">port</span> <span class="operator">=</span> ConfigUtils.getSystemProperty(protocolPrefix + key);</span><br><span class="line"> <span class="keyword">if</span> (port == <span class="literal">null</span> || port.length() == <span class="number">0</span>) {</span><br><span class="line"> port = ConfigUtils.getSystemProperty(key);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> port;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>也就是配置了<code>DUBBO_IP_TO_REGISTRY</code>这个环境变量</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>dubbo</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>记录一次折腾自组 nas 的失败经历-续篇</title>
|
|
<url>/2023/05/14/%E8%AE%B0%E5%BD%95%E4%B8%80%E6%AC%A1%E6%8A%98%E8%85%BE%E8%87%AA%E7%BB%84-nas-%E7%9A%84%E5%A4%B1%E8%B4%A5%E7%BB%8F%E5%8E%86-%E7%BB%AD%E7%AF%87/</url>
|
|
<content><![CDATA[<p>上次记录了前面的一些失败经验，最重要的点还没提到，先发一下配置单<br>cpu i7-8700k<br>主板 技嘉 z370m-ds3h<br>内存 光威 ddr4-3200Mhz<br>硬盘 京东京造 512g<br>散热 利民 PA120<br>电源 先马平头哥额定 550w<br>机箱 爱国者半岛铁盒 F10<br>cpu 跟主板是板 U 套装某鱼买的二手的，说实话如果不是后面的网卡问题，这个板 U 套装还是比较良心的，一次点亮（以前没组装过，还不知道有点不亮的情况，后面就体验到了），但是这里就出现了一个很大的坑，因为我这次是想要在裸机上装 exsi，然后看到了群里苏大的一篇 exsi 最新版本 8 的镜像构建文章，硬件也不是很旧，就想着用最新的系统，镜像写进 ventoy 后启动发现报错找不到网卡，这会我还没发现问题的严重性，想着按一些教程打个驱动进去就好了，而且我还以为驱动只要跟镜像 iso 放一块就行了，后面随着深入了解就知道要把驱动打进 iso 镜像里，但是找了一通发现我的网卡是瑞昱的 RTL8168，这个型号的板载网卡，走的是 PCIE 通道，有驱动的最后支持的系统是 exsi6.7，再往后就没有完整打包好的社区版驱动可以使用了，所以这是踩的第一个大坑，照理这个事情也没这么大问题，退回来 6.7 不就行了，问题恰恰是我那时候还不懂，又想用更新的系统，所以就在网上搜了半天，发现华硕的 z370 tuf gaming 系列是用的 intel 的网卡，社区的网卡驱动对 intel 的网卡支持比较好，所以想着还是换个主板算了，其实还有不少选择，买个 pcie 的 intel 网卡或者 usb 的其他千兆网卡，有个说出来可能比较难理解的，usb 的社区版驱动反而比 pcie 的支持得广，pcie 的还是只支持 intel 的。<br>在某多多上买了个二手的 z370 tuf gaming 主板，结果踩到了第二个坑，可能比较小白的经验是，前面因为买的板 U 套装，他 cpu 是直接装在主板上邮给我的，所以我没装过 cpu，这回买来这块二手的华硕主板对我来说是第一次装 cpu，不过好像难度不大，一下就装好了，但结果就很惨，就是点不亮，散热器风扇会转，但是键盘灯不亮，而且散热风扇还转得很快，我还试着把内存换个槽，结果四个槽都不行，这个时候就很害怕了，看上去这家店也不像是太坑的，毕竟大量地在卖，所以我就很担心是不是前面 cpu 装的不对，把针脚什么的搞坏了，这个时候已经搞到晚上很迟了，但还是忍不住又装回原来的技嘉主板试了下，幸好能正常点亮，算了，还是就用技嘉这块主板吧，接口配置稍微差了点，网卡也不支持最新版的 exsi，所以我就用 vmware workstation 了，在 win10 的 lstc 上装一个，有点性能损耗就损耗吧，反正我也不暴力使用，能跑跑其他 Ubuntu 虚拟机啥的就可以了，或者回到前面的结果，可以装 6.7 的，网上带了瑞昱网卡驱动的 exsi6.7 的镜像挺多的，可以自己打一个或者用别人打包好的。折腾不止踩坑不止呐。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>nas</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>nas</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>记录一次折腾自组 nas 的失败经历-续续篇</title>
|
|
<url>/2023/05/28/%E8%AE%B0%E5%BD%95%E4%B8%80%E6%AC%A1%E6%8A%98%E8%85%BE%E8%87%AA%E7%BB%84-nas-%E7%9A%84%E5%A4%B1%E8%B4%A5%E7%BB%8F%E5%8E%86-%E7%BB%AD%E7%BB%AD%E7%AF%87/</url>
|
|
<content><![CDATA[<p>之前这个机器已经算是跑起来了，虽然不是很完善也不是最佳实践，不过这篇可能也不算是失败经历了，因为最后成功跑起来了，在没法装最新版的 exsi 的情况下，并且我后面买的华硕 z370 主板点不亮，所以我也有点死心就直接用Windows 下装 vmware workstation 装虚拟机，然后直通硬盘来做 nas，这样可能对于其他人来说是很垃圾的方案，不过因为我很多常用软件的都是在 Windows 环境下的，并且纯黑裙的环境会比较浪费，相比一些同学在群晖里安装 Windows 虚拟机，我觉得还是反过来比较好，毕竟 vmware 做虚拟机应该是比群晖专业点，不过这个方案也有一些问题<br>第一种方式是直接找网上同学分享的处理好引导的 vmx，这种我碰到了一个问题就是在打开虚拟机安装群晖系统.pat的时候会提示“无法安装此文件，文件可能已损坏”，这其实不是真的文件已损坏，应该是群晖在做校验的时候存在什么条件没有通过，尝试了断网等方式都不成功，所以后来就用了比较釜底抽薪的方案，直接使用大佬开源的 arpl 引导制作工具<br>第二种方式一开始是躺在我 B 站收藏夹里，有个 <a href="https://space.bilibili.com/1215918387">up</a> 制作的，做得很细致，也把很多细节也解释了，过程其实不难，就是按步骤一步步执行，但是一开始选择了 918+的系统在我的方案里安装不了，会提示无法安装，经过视频下的评论的知道，尝试使用 920+的系统就顺利安装成功了，这里唯一的区别就是在添加硬盘的时候要选择物理磁盘，然后 vmware 给出的硬盘选项是Physical0， Physical1，记得别选错了，然后在启动后我租了 raid5,4 块 4T 的盘，可以组成一个 10T 多一点的存储空间，打算用来作为比较长读写的区域，更大的盘可能就会作为只读区域，减小写入量。后面还有一些问题待解决，一个是电源，考虑换个稍好一点，因为目前看下来电源风扇的噪音比较大，还有就是主板，最近看中了微星的 z390，不过价格比较贵，打算慢慢蹲蹲看。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>nas</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>nas</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>记录一次折腾自组 nas 的失败经历-续续续篇</title>
|
|
<url>/2023/06/18/%E8%AE%B0%E5%BD%95%E4%B8%80%E6%AC%A1%E6%8A%98%E8%85%BE%E8%87%AA%E7%BB%84-nas-%E7%9A%84%E5%A4%B1%E8%B4%A5%E7%BB%8F%E5%8E%86-%E7%BB%AD%E7%BB%AD%E7%BB%AD%E7%AF%87/</url>
|
|
<content><![CDATA[<p>最近在搞 nas 的时候又翻了个很大的错误，因为前面说了正在用的技嘉的 z370m, 这个主板是跟 cpu 一起买的，如果是只是用在 Windows 环境，没什么扩展要求，或者只用 6 个sata盘位，用一个 ssd 做系统盘，是挺不错的，但是如果是像我这样的要搞盘位比较多的nas，发现有一系列的不足点，<br>1、 前面有讲过，网卡问题，用的是 RTL 瑞昱的网卡，目前知道的最高能支持的 exsi 的版本是 6.7，再高的版本就不支持了，不过其实真的想用的话，6.7 版本也是可以用的，只是我当时是打包的 8.0 的系统，个人也不太喜欢用比较低版本的，所以算是一个不足点<br>2、 第二点是m.2插槽，只有一个m.2也是很难理解的，一个做系统盘，另一个可以作为简单的数组存储，或者作为群晖的下载缓存，如果只有一个的话特别是现在这个 ssd 的价格这么低了，加一个还是很香的，这个问题跟后面也是有点关系的<br>3、 PCIE 的插槽，这个主板只支持两个 PCIE 3.0 x1 的，它能支持 1Gbps 的传输速度，后面我买了个 PCIE 3.0 x1 转 m.2 的转接卡，这样其实真的是把速度拖慢了很多，还有一个 PCIE 3.0 X16 的，这是用来装显卡的，只用来扩展做个m.2或者 sata 又感觉很浪费，而且说不定未来会装个显卡<br>4、 风扇口的问题，风扇接口其实原来一直没概念，我在这次装 nas 之前其实自己没有完整装个台式机过，之前买的目前在使用的主力机两个风扇在换了的时候才知道是用的大 4pin 接口串接的，其中一个有 pwm 的接口也没用，这样就没办法用主板自带调速软件来进行自动或者手动调速，有时候比如晚上不关机，放在房间里风扇声很大还是挺吵的，这个主板只有两个风扇接口，一个是 cpu 的，一个是系统风扇，其实就只有一个可以用了<br>主要是以上几个点，所以我一直在关注二手主板，之前买了个华硕的 z370 tuf gaming，没想到点不亮，也不知道什么原因，还害我吓得以为 cpu 被我弄坏了，后面慢慢在网上看着发现有一块微星的 z390 TOMAHAWK战斧导弹，各方面接口都很不错，而且还有个千兆网口，在闲鱼蹲了很久终于在前不久蹲到了，买回来赶紧看看能不能点亮，结果一把点亮，还是用的螺丝刀开机的，就心里暗爽感觉捡到了宝，但是装进我的半岛铁盒 F10 机箱就发现跪了，出现的这个问题原来在网上哪里看到过，一直没注意，就是 sata 接口的方向，因为一个是 matx 的板，装进去即使 sata 接口的方向是跟主板平行的也留着足够的空间，但是 atx 板装上之后，大概只有 2 厘米左右的空间并且是两个叠着的端口，一个可能要能硬拗一下，但是两个我试了下，只能识别出一个，这个真让我翻了个大车，而且其实我换这个主板其实还有个大问题，就是我是在 Windows 下的vmware workstation 虚拟机里通过sata 直通，但是这个是认硬盘接口的，也就是顺序是有影响的，换了主板发现也不能支持，后面可能真的想用的话还得重新搞黑裙，要把数据备份出来，幸好换回原来的主板还能用，所以硬件也不能只看性能和接口丰富度，要看适配的其他硬件的合适度以及原先已经有的软件系统</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>nas</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>nas</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>记录一次折腾自组 nas 的失败经历</title>
|
|
<url>/2023/05/07/%E8%AE%B0%E5%BD%95%E4%B8%80%E6%AC%A1%E6%8A%98%E8%85%BE%E8%87%AA%E7%BB%84-nas-%E7%9A%84%E5%A4%B1%E8%B4%A5%E7%BB%8F%E5%8E%86/</url>
|
|
<content><![CDATA[<p>鉴于现在市面上的成品 nas 对于我来说要不就是太贵了，要不就是便宜的盘位少，性能比较差，很多 nas 主打还有用 docker 什么的，但是性能对于我个人比较特殊的使用方式来说还是不太够用的，比如现在比较性能好的 nas 像绿联新出的 DX4600，用的是N5105，passmark 分数还不如我 15 年买的 pc 上的 i5 4590，当然很多人是考虑功耗，这也是萝卜青菜各有所爱，可能我算下来还是觉得没多大必要<br>然后就是考虑用什么硬件配置了，这个流派也有很多种，<br>用蜗牛星际的原版硬件其实对于需求不大的也是挺好的，整套的都解决了，cpu 用 j1900 如果就做 nas 应该也够了，我没选的原因一方面是性能不符合我的要求，另一方面是现在市面上的机器大部分都是战损成色，而且也不太便宜，如果成色比较好的能够 400 以内拿下整机的话感觉还算可以，cpu 换成 j4125 或者 j3455 再加个 100 也能接受，但基本比较少有这种价格，之前看到一个换了 j3455 的只要 360，犹豫了下没下手，其他很多的都是 j1900 的都要 600 左右<br>然后是各类 E3，E5 和商用服务器类型的，这种的特点是功耗大，其实 cpu 很便宜，E5 2630V3 跟 2630V4 都只要十几块钱，性能过得去，没有选的主要是机箱占地方也比较贵，还有是商用的怕很多系统需要自己找驱动什么的，配件比如 HBA 卡这种，买的不兼容什么的还是挺麻烦的，另外噪音也是个比较大的问题，租的房子比较小，即使放客厅也是靠着卧室的墙边，如果以后换大一点的房子倒是可以考虑<br>最后就是我目前选的方案，就是普通的民用机器，找盘位多一点的机箱，我原来的 4590 的机器的机箱就不错，但是已经停产了，二手的太重了闲鱼都不出外地，cpu 跟主板其实考虑了很久，因为从 4590 开始核显就能硬解 H264 这种常规的视频了，考虑用intel 四代的 i3 或者 i5 应该纯 nas 来讲是足够用了，但是这样就跟我现在已经有的 4590 有点重叠了，而且也觉得最好是能性能好一些的，就开始看一些稍新一点的，很多用的多的有 i3-8100，跟 i3-10100 这种，但是这些已经被炒的价格比较高，寻寻觅觅了很久看中了 i5-8600，这个价格跟 8500 差不多，性能还好一些，主板就是我标题说的最“失败”的一个点了，主要是因为主板自带的网卡是 Realtek 的，至于更具体的后面会专门介绍，机箱是图便宜买的爱国者半岛铁盒的 F10，内存就买了一根光威的 32g 的，装系统的硬盘是用了以前囤的京东京造的麒麟系列，但是现在对这个系列挺不看好，之前有个盘就掉盘了，维修体验一开始也不好，半个多月维修，电源也是图便宜买的一个先马的平头哥系列，额定 550w 只要 140 左右，整体的机器就攒齐了，但是很多问题也随之出现了<br>第一个问题是买的二手的板 U 套装，结果寄过来的时候没带挡板，导致一开始装上了又要拆下来；第二个问题是主机贪便宜，主机上固定主板螺丝的螺柱拧不进去，后来店家告诉我让我可以用电源固定的螺丝先拧一下才把螺柱拧进去；第三个问题是因为没什么装机经验导致的，散热装的太累了，因为要兼容各种主板，还有各种螺丝，装的时候也着急；第四个问题是电源的比较便宜，一方面比较不放心安全性，另一方面是我想多装几个硬盘，电源直出的只有四个 sata 口，需要买转接线，从 D 型的 4pin 口子转出来；第五个问题也是电源，主板电源线有点不太够，走背线就比较困难<br>以上主要是装机的困难，下一篇介绍作为 nas 的各种问题吧</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>nas</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>nas</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>记录下 Java Stream 的一些高效操作</title>
|
|
<url>/2022/05/15/%E8%AE%B0%E5%BD%95%E4%B8%8B-Java-Lambda-%E7%9A%84%E4%B8%80%E4%BA%9B%E9%AB%98%E6%95%88%E6%93%8D%E4%BD%9C/</url>
|
|
<content><![CDATA[<p>我们日常在代码里处理一些集合逻辑的时候用到 Stream 其实还挺多的，普通的取值过滤集合一般都是结合 ide 的提示就能搞定了，但是有些不太常用的就在这记录下，争取后面都更新记录下来。</p>
|
|
<h3 id="自定义-distinctByKey-对结果进行去重"><a href="#自定义-distinctByKey-对结果进行去重" class="headerlink" title="自定义 distinctByKey 对结果进行去重"></a>自定义 distinctByKey 对结果进行去重</h3><p>stream 中自带的 distinct 只能对元素进行去重<br>比如下面代码</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> List<Integer> list = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"> list.add(<span class="number">1</span>);</span><br><span class="line"> list.add(<span class="number">1</span>);</span><br><span class="line"> list.add(<span class="number">2</span>);</span><br><span class="line"> list = list.stream().distinct().collect(Collectors.toList());</span><br><span class="line"> System.out.println(list);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>结果就是去了重的</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">[<span class="number">1</span>, <span class="number">2</span>]</span><br></pre></td></tr></table></figure>
|
|
<p>但是当我的元素是个复杂对象，我想根据对象里的某个元素进行过滤的时候，就需要用到自定义的 <code>distinctByKey</code> 了，比如下面的想对 userId 进行去重</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> {</span><br><span class="line"> List<StudentRecord> list = <span class="keyword">new</span> <span class="title class_">ArrayList</span><>();</span><br><span class="line"> <span class="type">StudentRecord</span> <span class="variable">s1</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">StudentRecord</span>();</span><br><span class="line"> s1.setUserId(<span class="number">11L</span>);</span><br><span class="line"> s1.setCourseId(<span class="number">100L</span>);</span><br><span class="line"> s1.setScore(<span class="number">100</span>);</span><br><span class="line"> list.add(s1);</span><br><span class="line"> <span class="type">StudentRecord</span> <span class="variable">s2</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">StudentRecord</span>();</span><br><span class="line"> s2.setUserId(<span class="number">11L</span>);</span><br><span class="line"> s2.setCourseId(<span class="number">101L</span>);</span><br><span class="line"> s2.setScore(<span class="number">100</span>);</span><br><span class="line"> list.add(s2);</span><br><span class="line"> <span class="type">StudentRecord</span> <span class="variable">s3</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">StudentRecord</span>();</span><br><span class="line"> s3.setUserId(<span class="number">12L</span>);</span><br><span class="line"> s3.setCourseId(<span class="number">100L</span>);</span><br><span class="line"> s3.setScore(<span class="number">100</span>);</span><br><span class="line"> list.add(s3);</span><br><span class="line"> System.out.println(list.stream().distinct().collect(Collectors.toList()));</span><br><span class="line"> }</span><br><span class="line"> <span class="meta">@Data</span></span><br><span class="line"> <span class="keyword">static</span> <span class="keyword">class</span> <span class="title class_">StudentRecord</span> {</span><br><span class="line"> Long id;</span><br><span class="line"> Long userId;</span><br><span class="line"> Long courseId;</span><br><span class="line"> Integer score;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>结果就是</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">[StudentRecord(id=<span class="literal">null</span>, userId=<span class="number">11</span>, courseId=<span class="number">100</span>, score=<span class="number">100</span>), StudentRecord(id=<span class="literal">null</span>, userId=<span class="number">11</span>, courseId=<span class="number">101</span>, score=<span class="number">100</span>), StudentRecord(id=<span class="literal">null</span>, userId=<span class="number">12</span>, courseId=<span class="number">100</span>, score=<span class="number">100</span>)]</span><br></pre></td></tr></table></figure>
|
|
<p>因为对象都不一样，所以就没法去重了，这里就需要用</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <T> Predicate<T> <span class="title function_">distinctByKey</span><span class="params">(</span></span><br><span class="line"><span class="params"> Function<? <span class="built_in">super</span> T, ?> keyExtractor)</span> {</span><br><span class="line"></span><br><span class="line"> Map<Object, Boolean> seen = <span class="keyword">new</span> <span class="title class_">ConcurrentHashMap</span><>();</span><br><span class="line"> <span class="keyword">return</span> t -> seen.putIfAbsent(keyExtractor.apply(t), Boolean.TRUE) == <span class="literal">null</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
|
|
<p>然后就可以用它来去重了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">System.out.println(list.stream().filter(distinctByKey(StudentRecord::getUserId)).collect(Collectors.toList()));</span><br></pre></td></tr></table></figure>
|
|
<p>看下结果</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">[StudentRecord(id=<span class="literal">null</span>, userId=<span class="number">11</span>, courseId=<span class="number">100</span>, score=<span class="number">100</span>), StudentRecord(id=<span class="literal">null</span>, userId=<span class="number">12</span>, courseId=<span class="number">100</span>, score=<span class="number">100</span>)]</span><br></pre></td></tr></table></figure>
|
|
<p>但是说实在的这个功能感觉应该是 stream 默认给实现的</p>
|
|
<h3 id="使用-java-util-stream-Collectors-groupingBy-对-list-进行分组"><a href="#使用-java-util-stream-Collectors-groupingBy-对-list-进行分组" class="headerlink" title="使用 java.util.stream.Collectors#groupingBy 对 list 进行分组"></a>使用 java.util.stream.Collectors#groupingBy 对 list 进行分组</h3><p>这个使用场景还是蛮多的，上面的场景里比如我要对 userId 进行分组，就一行代码就解决了</p>
|
|
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">System.out.println(list.stream().collect(Collectors.groupingBy(StudentRecord::getUserId)));</span><br></pre></td></tr></table></figure>
|
|
<h4 id="结果"><a href="#结果" class="headerlink" title="结果"></a>结果</h4><figure class="highlight java"><table><tr><td class="code"><pre><span class="line">{<span class="number">11</span>=[StudentRecord(id=<span class="literal">null</span>, userId=<span class="number">11</span>, courseId=<span class="number">100</span>, score=<span class="number">100</span>), StudentRecord(id=<span class="literal">null</span>, userId=<span class="number">11</span>, courseId=<span class="number">101</span>, score=<span class="number">100</span>)], <span class="number">12</span>=[StudentRecord(id=<span class="literal">null</span>, userId=<span class="number">12</span>, courseId=<span class="number">100</span>, score=<span class="number">100</span>)]}</span><br></pre></td></tr></table></figure>
|
|
<p>很方便的变成了以 <code>userId</code> 作为 <code>key</code>，以相同 <code>userId</code> 的 <code>StudentRecord</code> 的 <code>List</code> 作为 <code>value</code> 的 <code>map</code> 结构</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>java</tag>
|
|
<tag>stream</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>记录下 phpunit 的入门使用方法</title>
|
|
<url>/2022/10/16/%E8%AE%B0%E5%BD%95%E4%B8%8B-phpunit-%E7%9A%84%E5%85%A5%E9%97%A8%E4%BD%BF%E7%94%A8%E6%96%B9%E6%B3%95/</url>
|
|
<content><![CDATA[<p>这周开始打算写个比较简单的php工具包，然后顺带学习使用下php的单元测试，通过phpunit还是比较方便的，首先就<code>composer require phpunit/phpunit</code><br>安装下 <code>phpunit</code>, 前面包就是通过 <code>composer init</code> 创建，装完依赖后就可以把自动加载代码生成下 <code>composer dump-autoload</code><br>目录结构差不多这样</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">.</span><br><span class="line">├── composer.json</span><br><span class="line">├── composer.lock</span><br><span class="line">├── oldfile.txt</span><br><span class="line">├── phpunit.xml</span><br><span class="line">├── src</span><br><span class="line">│ └── Rename.php</span><br><span class="line">└── tests</span><br><span class="line"> └── RenameTest.php</span><br><span class="line"></span><br><span class="line">2 directories, 6 files</span><br></pre></td></tr></table></figure>
|
|
<p><code>src/</code>是源码，<code>tests/</code>是放的单测，比较重要的是<code>phpunit.xml</code></p>
|
|
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="meta"><?xml version=<span class="string">"1.0"</span> encoding=<span class="string">"UTF-8"</span>?></span></span><br><span class="line"><span class="tag"><<span class="name">phpunit</span> <span class="attr">colors</span>=<span class="string">"true"</span> <span class="attr">bootstrap</span>=<span class="string">"vendor/autoload.php"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">testsuites</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">testsuite</span> <span class="attr">name</span>=<span class="string">"php-rename"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">directory</span>></span>./tests/<span class="tag"></<span class="name">directory</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">testsuite</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">testsuites</span>></span></span><br><span class="line"><span class="tag"></<span class="name">phpunit</span>></span></span><br></pre></td></tr></table></figure>
|
|
<p>其中bootstrap就是需要把依赖包的自动加载入口配上，因为这个作为一个package，也会指出命名空间<br>然后就是testsuite的路径，源码中</p>
|
|
<figure class="highlight php"><table><tr><td class="code"><pre><span class="line"><span class="meta"><?php</span></span><br><span class="line"><span class="keyword">namespace</span> <span class="title class_">Nicksxs</span>\<span class="title class_">PhpRename</span>;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Rename</span></span></span><br><span class="line"><span class="class"></span>{</span><br><span class="line"> <span class="keyword">public</span> <span class="built_in">static</span> <span class="function"><span class="keyword">function</span> <span class="title">renameSingleFile</span>(<span class="params"><span class="variable">$file</span>, <span class="variable">$newFileName</span></span>): <span class="title">bool</span></span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="keyword">if</span>(!<span class="title function_ invoke__">is_file</span>(<span class="variable">$file</span>)) {</span><br><span class="line"> <span class="keyword">echo</span> <span class="string">"it's not a file"</span>;</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="variable">$fileInfo</span> = <span class="title function_ invoke__">pathinfo</span>(<span class="variable">$file</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="title function_ invoke__">rename</span>(<span class="variable">$file</span>, <span class="variable">$fileInfo</span>[<span class="string">"dirname"</span>] . DIRECTORY_SEPARATOR . <span class="variable">$newFileName</span> . <span class="string">"."</span> . <span class="variable">$fileInfo</span>[<span class="string">"extension"</span>]);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>就是一个简单的重命名<br>然后test代码是这样，</p>
|
|
<figure class="highlight php"><table><tr><td class="code"><pre><span class="line"><span class="meta"><?php</span></span><br><span class="line"></span><br><span class="line"><span class="comment">// require_once 'vendor/autoload.php';</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">use</span> <span class="title">PHPUnit</span>\<span class="title">Framework</span>\<span class="title">TestCase</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">Nicksxs</span>\<span class="title">PhpRename</span>\<span class="title">Rename</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="keyword">function</span> <span class="title">PHPUnit</span>\<span class="title">Framework</span>\<span class="title">assertEquals</span>;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">RenameTest</span> <span class="keyword">extends</span> <span class="title">TestCase</span> </span></span><br><span class="line"><span class="class"></span>{</span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">setUp</span>(<span class="params"></span>) :<span class="title">void</span></span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="variable">$myfile</span> = <span class="title function_ invoke__">fopen</span>(<span class="keyword">__DIR__</span> . DIRECTORY_SEPARATOR . <span class="string">"oldfile.txt"</span>, <span class="string">"w"</span>) <span class="keyword">or</span> <span class="keyword">die</span>(<span class="string">"Unable to open file!"</span>);</span><br><span class="line"> <span class="variable">$txt</span> = <span class="string">"file test1\n"</span>;</span><br><span class="line"> <span class="title function_ invoke__">fwrite</span>(<span class="variable">$myfile</span>, <span class="variable">$txt</span>);</span><br><span class="line"> <span class="title function_ invoke__">fclose</span>(<span class="variable">$myfile</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">testRename</span>(<span class="params"></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="title class_">Rename</span>::<span class="title function_ invoke__">renameSingleFile</span>(<span class="keyword">__DIR__</span> . DIRECTORY_SEPARATOR . <span class="string">"oldfile.txt"</span>, <span class="string">"newfile"</span>);</span><br><span class="line"> <span class="title function_ invoke__">assertEquals</span>(<span class="title function_ invoke__">is_file</span>(<span class="keyword">__DIR__</span> . DIRECTORY_SEPARATOR . <span class="string">"newfile.txt"</span>), <span class="literal">true</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="function"><span class="keyword">function</span> <span class="title">tearDown</span>(<span class="params"></span>): <span class="title">void</span></span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="title function_ invoke__">unlink</span>(<span class="keyword">__DIR__</span> . DIRECTORY_SEPARATOR . <span class="string">"newfile.txt"</span>);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p><code>setUp</code> 跟 <code>tearDown</code> 就是初始化跟结束清理的，但是注意如果不指明 <code>__DIR__</code> ,待会的目录就会在执行 <code>vendor/bin/phpunit</code> 下面，<br>或者也可以指定在一个 <code>tmp/</code> 目录下<br>最后就可以通过<code>vendor/bin/phpunit</code> 来执行测试<br>执行结果</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">❯ vendor/bin/phpunit</span><br><span class="line">PHPUnit 9.5.25 by Sebastian Bergmann and contributors.</span><br><span class="line"></span><br><span class="line">. 1 / 1 (100%)</span><br><span class="line"></span><br><span class="line">Time: 00:00.005, Memory: 6.00 MB</span><br><span class="line"></span><br><span class="line">OK (1 test, 1 assertion)</span><br></pre></td></tr></table></figure>
|
|
]]></content>
|
|
<categories>
|
|
<category>php</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>php</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>记录下 phpunit 的入门使用方法之setUp和tearDown</title>
|
|
<url>/2022/10/23/%E8%AE%B0%E5%BD%95%E4%B8%8B-phpunit-%E7%9A%84%E5%85%A5%E9%97%A8%E4%BD%BF%E7%94%A8%E6%96%B9%E6%B3%95%E4%B9%8BsetUp%E5%92%8CtearDown/</url>
|
|
<content><![CDATA[<p>可能是太久没写单测了，写个单测发现不符合预期，后来验证下才反应过来<br>我们来看下demo</p>
|
|
<figure class="highlight php"><table><tr><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">RenameTest</span> <span class="keyword">extends</span> <span class="title">TestCase</span></span></span><br><span class="line"><span class="class"></span>{</span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">setUp</span>(<span class="params"></span>): <span class="title">void</span></span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="title function_ invoke__">var_dump</span>(<span class="string">"setUp"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">test1</span>(<span class="params"></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="title function_ invoke__">var_dump</span>(<span class="string">"test1"</span>);</span><br><span class="line"> <span class="title function_ invoke__">assertEquals</span>(<span class="number">1</span>, <span class="number">1</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">test2</span>(<span class="params"></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="title function_ invoke__">var_dump</span>(<span class="string">"test2"</span>);</span><br><span class="line"> <span class="title function_ invoke__">assertEquals</span>(<span class="number">1</span>, <span class="number">1</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">protected</span> <span class="function"><span class="keyword">function</span> <span class="title">tearDown</span>(<span class="params"></span>): <span class="title">void</span></span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="title function_ invoke__">var_dump</span>(<span class="string">"tearDown"</span>);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
|
|
<p>因为我是想写个重命名的小工具，希望通过<code>setUp</code>和<code>tearDown</code>做一些文件初始化和清理工作，但是我把两个case的初始化跟清理工作写到了单个<code>setUp</code>和<code>tearDown</code>中，这样就出现了异常的错误<br>通过上面的示例代码，可以看到执行结果</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">❯ vendor/bin/phpunit</span><br><span class="line">PHPUnit 9.5.25 by Sebastian Bergmann and contributors.</span><br><span class="line"></span><br><span class="line">.string(5) "setUp"</span><br><span class="line">string(5) "test1"</span><br><span class="line">string(8) "tearDown"</span><br><span class="line">. 2 / 2 (100%)string(5) "setUp"</span><br><span class="line">string(5) "test2"</span><br><span class="line">string(8) "tearDown"</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">Time: 00:00.005, Memory: 6.00 MB</span><br><span class="line"></span><br><span class="line">OK (2 tests, 2 assertions)</span><br></pre></td></tr></table></figure>
|
|
<p>其实就是很简单的会在每个test方法前后都执行<code>setUp</code>和<code>tearDown</code></p>
|
|
]]></content>
|
|
<categories>
|
|
<category>php</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>php</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>记录下 redis 的一些使用方法</title>
|
|
<url>/2022/10/30/%E8%AE%B0%E5%BD%95%E4%B8%8B-redis-%E7%9A%84%E4%B8%80%E4%BA%9B%E4%BD%BF%E7%94%A8%E6%96%B9%E6%B3%95/</url>
|
|
<content><![CDATA[<p>虽然说之前讲解过一些redis源码相关的，但是说实话，redis的各种使用其实有时候有点生疏，或者在一些特定的使用场景中，一些使用方法还是需要学习和记录的</p>
|
|
<h3 id="获取所有数据"><a href="#获取所有数据" class="headerlink" title="获取所有数据"></a>获取所有数据</h3><p>获取<code>list</code>类型的所有元素，可以使用 <a href="https://redis.io/commands/lrange/">lrange</a> , 直接用<code>lrange key 0 -1</code><br>比如<br><img data-src="https://img.nicksxs.me/uPic/WechatIMG1082.png"><br>这里有一些方便的就是可以不用知道长度，直接全返回，或者如果想拿到特定区间的就可以直接指定起止范围，<br><img data-src="https://img.nicksxs.me/blog/YRpfWA.png"><br>这样就不用一个个<code>pop</code>出来</p>
|
|
<h3 id="裁剪list"><a href="#裁剪list" class="headerlink" title="裁剪list"></a>裁剪list</h3><p>前面用了<code>lrange</code>取得了一个范围的数据，如果想将数据直接移除，那可以用 <a href="https://redis.io/commands/ltrim/">ltrim</a> ,<br><img data-src="https://img.nicksxs.me/blog/0607zR.png"><br>这两个命令就可以从<code>list</code>里取出批量数据，并且能从<code>list</code>里删除这部分数据</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>redis</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>redis</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>记录下 zookeeper 集群迁移和易错点</title>
|
|
<url>/2022/05/29/%E8%AE%B0%E5%BD%95%E4%B8%8B-zookeeper-%E9%9B%86%E7%BE%A4%E8%BF%81%E7%A7%BB/</url>
|
|
<content><![CDATA[<p>前阵子做了zk 的集群升级迁移，大概情况是原来是一个三节点的 zk 集群（最小可用<br>大概是</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">zk1 192.168.2.1</span><br><span class="line">zk2 192.168.2.2</span><br><span class="line">zk3 192.168.2.3</span><br></pre></td></tr></table></figure>
|
|
<p>在 zoo.cfg 中的配置就是如下</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">server.1=192.168.2.1:2888:3888</span><br><span class="line">server.2=192.168.2.2:2888:3888</span><br><span class="line">server.3=192.168.2.3:2888:3888</span><br></pre></td></tr></table></figure>
|
|
<h2 id="加节点"><a href="#加节点" class="headerlink" title="加节点"></a>加节点</h2><p>需要将集群迁移到 192.168.2.4（简称 zk4），192.168.2.5（简称 zk5），192.168.2.6（简称 zk6） 这三台机器上，目前新的这三台机器上是没有 zk 部署的, 我们想要的是数据不丢失，那主要考虑的就是滚动升级，这里我其实犯了几个错误，也特别说明下<br>首先我们想要新的三台机器加进去，所以我在zk4，zk5，zk6 的配置是这样</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">server.1=192.168.2.1:2888:3888</span><br><span class="line">server.2=192.168.2.2:2888:3888</span><br><span class="line">server.3=192.168.2.3:2888:3888</span><br><span class="line">server.4=192.168.2.4:2888:3888</span><br><span class="line">server.5=192.168.2.5:2888:3888</span><br><span class="line">server.6=192.168.2.6:2888:3888</span><br></pre></td></tr></table></figure>
|
|
<p>这样起来发现状态是该节点没起来，<br>PS：查看当前节点状态可以通过 <code>./zkServer.sh status</code> 来查看<br>第一个问题是我需要一个<code>myid</code>文件，标识我是哪个节点，里面的内容就写 <code>4</code>或 <code>5</code> 或 <code>6</code> 这样就行了，并且这个文件的路径应该在配置文件中指定的<code>dataDir=</code>数据目录下<br>第二个问题是困扰我比较久的，我在按上面的配置启动节点后，发现这几个节点都是没起来的，并且有 <code>FastLeaderElection@xxx - Notification time out: 60000</code> 这个报错，一开始以为是网络不通，端口没开这些原因，检查了下都是通的，结果原因其实跟我之前的一个考虑是相关的，当有六个节点的时候，理论上需要有半数以上的节点可用，集群才会是健康的，但是按我这个方式起来，其实我配置了六个节点，但是其中三个都是不可用的（包括自身节点），那么它自然是没办法正常工作，所以这里其实也需要滚动添加，类似于这样<br>我的 <code>zk4</code> 的配置应该是这样</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">server.1=192.168.2.1:2888:3888</span><br><span class="line">server.2=192.168.2.2:2888:3888</span><br><span class="line">server.3=192.168.2.3:2888:3888</span><br><span class="line">server.4=192.168.2.4:2888:3888</span><br></pre></td></tr></table></figure>
|
|
<p>然后 <code>zk5</code> 的配置</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">server.1=192.168.2.1:2888:3888</span><br><span class="line">server.2=192.168.2.2:2888:3888</span><br><span class="line">server.3=192.168.2.3:2888:3888</span><br><span class="line">server.4=192.168.2.4:2888:3888</span><br><span class="line">server.5=192.168.2.5:2888:3888</span><br></pre></td></tr></table></figure>
|
|
<p>接着 <code>zk6</code> 的配置就可以是全部了</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">server.1=192.168.2.1:2888:3888</span><br><span class="line">server.2=192.168.2.2:2888:3888</span><br><span class="line">server.3=192.168.2.3:2888:3888</span><br><span class="line">server.4=192.168.2.4:2888:3888</span><br><span class="line">server.5=192.168.2.5:2888:3888</span><br><span class="line">server.6=192.168.2.6:2888:3888</span><br></pre></td></tr></table></figure>
|
|
<p>然后为了集群完全更新，就继续在 <code>zk4</code> 和 <code>zk5</code> 加上其他节点，这样我的 6 节点集群就起来了</p>
|
|
<h2 id="下节点"><a href="#下节点" class="headerlink" title="下节点"></a>下节点</h2><p>这里我踩了另外一个坑，或者说没搞清楚两种方式的差别，</p>
|
|
<h3 id="第一种"><a href="#第一种" class="headerlink" title="第一种"></a>第一种</h3><p>首先说说我没采用的第一种方式，（也是比较合理的）其实上面这个集群有个明显的问题，老集群其实还是各自认了一个三节点的集群，其中 zk3 是主节点，对于 zk1，zk2，zk3 来说它们能看到的就只有这三个节点，对于后三个 zk4，zk5，zk6 节点来说他们能连上其余五个节点，可以认为这是个六节点的集群，那么比较合理的操作应该是在老的三节点上把后面三个也都加进来，即每个节点的配置里 server 都有 6 个，然后我再对老的节点进行下线，这里下线需要注意的比较理想的是下一个节点就要修改配置，挪掉下线的节点后进行一遍重启，比如我知道了集群中的 leader 是在 zk3 上面，那么我先将 zk1 和 zk2 下掉，那么在我将 zk1 下线的之后，我将其他的五个节点都删除 zk1 的配置，然后重启，这样其实不是必须，但相对会可靠些，理论上我也可以在下掉 zk1 和 zk2 之后再修改配置重启其余节点。而当只剩下 zk3，zk4，zk5，zk6 四个节点的集群后，并且每个节点里的配置也只有这四个 server，我再下线 zk3 这个 leader 的时候，就会进行选举，再选出新的 leader，因为刚好是三节点，同样保证了最小可用。</p>
|
|
<h3 id="第二种"><a href="#第二种" class="headerlink" title="第二种"></a>第二种</h3><p>这也是我踩坑的一种方式，就是我没有修改原来三节点的配置，并且我一开始以为可以通过下线 zk1，zk2，zk3（进行选举）的方式完成下线，然后再进行重启，但是这种方式就是我上面说的，原来的三节点里我下掉 zk1 还是能够正常运行，但是我下线 zk2 的时候，这个集群就等于是挂了，小于最小可用了，这样三节点都挂了，而且对于新加入的三个节点来说，又回到了最初起不来一样状态，六节点里只有三节点在线，导致整个集群都挂了，所以对于我这样的操作来说，我需要滚动修改启动，在下线 zk1 的时候就需要把 zk4，zk5，zk6 中的 zk1 移除后重启，当然这样唯一的好处就是可以少重启几个，同样继续下线 zk2 的时候，把 zk2 移除掉再重启，其实在移除 zk1 后修改重启后，在下线 zk2 的时候，集群就会重新选举了，因为 zk2 下线的时候，zk3 还是会一起下线。这个是我们需要特别注意的</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>java</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>zookeeper</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>记录下一次服务器迁移</title>
|
|
<url>/2023/11/12/%E8%AE%B0%E5%BD%95%E4%B8%8B%E4%B8%80%E6%AC%A1%E6%9C%8D%E5%8A%A1%E5%99%A8%E8%BF%81%E7%A7%BB/</url>
|
|
<content><![CDATA[<p>因为服务器续费即使是这次双十一还是太贵了，之前在 tx 买的三年 2c4g 3m 的一台服务器，原来价格是三年 800+，现在续费三年要 3700，因为之前还买了其他的服务器，所以感觉再这么贵地续一个有点不划算就考虑迁移了服务器<br>主要是分为四块内容，先大致记录下，后续会慢慢展开讲<br>第一部分就是 headscale 的迁移，本身 headscale 的部署其实还好，原来的服务器是因为权限搞得几个文件都有问题，这次就按着第一次已经修改好的文件进行部署就好了，问题不大，可以参考 <a href="https://nicksxs.me/2023/01/22/Headscale%E5%88%9D%E4%BD%93%E9%AA%8C%E4%BB%A5%E5%8F%8A%E8%B8%A9%E5%9D%91%E8%AE%B0/">Headscale初体验以及踩坑记
|
|
</a>,主要的一点是要考虑 derper 的部署，以及客户端认证逻辑<br>第二部分是 traefik 以及背后的 gitea，WordPress 的迁移，这个的部署方式都是使用 docker-compose 的，已经算比较方便了的，但是要注意两点，第一个是对于这些有状态的需要先停止 docker-compose ，然后在进行数据迁移，第二步数据迁移可能就需要使用到 docker 命令，执行 mysqldump<br>如</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">docker exec -it mysql_server【docker容器名称/ID】 mysqldump -uroot -p123456【数据库密码】 test_db【数据库名称】 > /opt/sql_bak/test_db.sql【导出表格路径】</span><br></pre></td></tr></table></figure>
|
|
<p>然后再在新的 mysql 容器中先 cp 到容器内，然后在 docker 内用 mysql 命令连接 mysql-server 以后用 source 将 sql 导进来<br>而像 traefik 本身其实是无状态的，只需要配置文件没有什么机器依赖就好，还有就是证书的更新，最后就是域名解析问题，修改解析就好<br>第三部分是 rustdesk 的迁移，这个其实就按官方文档来跑两个容器就好，hbbr 跟 hbbs 就好了，一个是 relay，一个是 id 服务器，不过这里有个问题我目前没看到怎么识别是否已经用到了这个，有种部署了但是不知道有没有用的尴尬，有知道的可以指导下<br>第四部分是 qinglong 的部署，这个也挺简单了，就是有个小问题，如果部署的订阅链接有 github 的就可以去站长的 ping 连接速度看看，自己配个 hosts 就会比较方便，不然经常容易拉不下来，还有就是要注意安全，至少要改强密码</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>与服务器</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>云服务器</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>记录下把小米路由器 4A 千兆版刷成 openwrt 的过程</title>
|
|
<url>/2023/05/21/%E8%AE%B0%E5%BD%95%E4%B8%8B%E6%8A%8A%E5%B0%8F%E7%B1%B3%E8%B7%AF%E7%94%B1%E5%99%A8-4A-%E5%8D%83%E5%85%86%E7%89%88%E5%88%B7%E6%88%90-openwrt-%E7%9A%84%E8%BF%87%E7%A8%8B/</url>
|
|
<content><![CDATA[<p>之前在绍兴家里的一条宽带送了个小米路由器 4A，正好原来的小米路由器 3 不知道为啥经常断流不稳定，而且只支持百兆，这边用了 200M 的宽带，感觉也比较浪费，所以就动了这个心思，但是还是有蛮多坑的，首先是看到了一篇文章，写的比较详细，<br>看到的就是这篇<a href="https://www.cwlog.net/archives/553.html">文章</a><br>这里使用的是 <a href="https://github.com/acecilia/OpenWRTInvasion/releases">OpenWRTInvasion</a> 这个项目来破解 ssh，首先这里有个最常见的一个问题，就是文件拉不到，所以有一些可行的方法就是自己起一个http 服务，可以修改脚本代码，直接从这个启动的 http 服务拉取已经下载下的文件，就这个问题我就尝试了很多次，还有就是这个 OpenWRTInvasion 最后一个支持 Windows 的版本就是 0.0.7，后面的版本其实做了很多的优化解决了文件的问题，一开始碰到的问题是本地起了文件服务但是没请求，或者请求了但后续 ssh 没有正常破解，我就换了 Mac 用最新版本的OpenWRTInvasion来尝试进行破解，发现还是不行，结果查了不少资料发现最根本的问题是这个路由器的新版本就不支持这种破解了，因为这个路由器新的版本都是 v2 版本，也就是2.30.x 版本的系统了，原来支持的是 2.28.x 的这些系统，后来幸好是找到了这个版本的系统支持的另一个恩山大神的<a href="https://www.right.com.cn/forum/thread-8261014-1-1.html">文章</a>，根据这个文章提供的工具进行破解就成功了，但是破解要多尝试几次，我第一次是失败的，小米路由器 4A 千兆版的版本号也会写作 R4Av2，在搜索一些资料的时候也可以用这个型号去搜，可能也是另一种黑话，路由器以前刷过梅林，padavan，还是第一次刷 openwrt，都已经忘了以前是怎么刷的来着，感觉现在越来越难刷了，特别是 ssh，想给我的 ax6 刷个 openwrt，发现前提是需要先有一个 openwrt 的路由器，简直了，变成先有鸡还是先有蛋的问题了，所以我把这个小米 4A 刷成 openwrt 也有这个考虑，毕竟 4A 配置上不太高，openwrt 各种插件可能还跑不起来，权当做练手和到时候用来开 AX6 的工具了。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>路由器</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>这周末我又在老丈人家打了天小工</title>
|
|
<url>/2020/08/30/%E8%BF%99%E5%91%A8%E6%9C%AB%E6%88%91%E5%8F%88%E5%9C%A8%E8%80%81%E4%B8%88%E4%BA%BA%E5%AE%B6%E6%89%93%E4%BA%86%E5%A4%A9%E5%B0%8F%E5%B7%A5/</url>
|
|
<content><![CDATA[<p>因为活实在比较多，也不太好叫大工(活比较杂散)，相比上一次我跟 LD 俩人晚起了一点，我真的是只要有事，早上就醒的很早，准备八点出发的，六点就醒了，然后想继续睡就一直做梦🤦♂️，差不多八点半多到的丈人家，他们应该已经干了有一会了，我们到了以后就分配给我撬地板的活，上次说的那个敲掉柜子的房间里，还铺着质地还不错的木地板，但是也不想要了，得撬掉重新铺。<br>拿着撬棍和榔头就上楼去干了，浙江这几天的天气，最高温度一般 38、9，楼上那个房间也没风扇，有了也不能用，都是灰尘，撬了两下，我感觉我体内的水就像真气爆发一样变成汗炸了出来，眼睛全被汗糊住了，可能大部分人不太了解地板是怎么铺的，一般是在地面先铺一层混凝土，混凝土中间嵌进去规则的长条木条，然后真正的地板一块块的都是钉在那个木条上，用那种气枪钉和普通的钉子，并且块跟块之前还有一个木头的槽结构相互耦合，然后边缘的一圈在用较薄的木板将整个木地板封边(这些词都是我现造的)，边缘的用的钉子会更多，所以那几下真的很用力，而且撬地板，得蹲下起来，如此反复，对于我这个体重快超过身高的中年人来说的确是非常大的挑战，接下来继续撬了几个，已经有种要虚脱晕倒的感觉了，及时去喝水擦了汗，又歇了一会，为啥一上来就这么拼呢，主要是因为那个房间丈人在干活的时候是直接看得到的🤦♂️，后来被 LD 一顿教育，本来就是去帮忙的，又不是专业做这个的，急啥。<br>喝了水之后，又稍稍歇了一会，就开始继续撬了，本来觉得这个地板撬着好像还行，房间不大，没多久就撬完了，撬完之后喝了点饮料(补充点糖分，早餐吃得少，有点低血糖)，然后看到 LD 在撬下面的木条了，这个动作开始了那天最大的经验值收集行动，前面说了这个木条一般是跟混凝土一块铺上去的，但是谁也没想到，这个混凝土铺上去的时候竟然处理的这么随意，根本没考虑跟下面的贴合，所以撬木条的时候直接把木条跟木条中间大块大块的混凝土一块撬起来了，想想那重量，于是我这靠蛮力干活的，就用力把木条带着混凝土一块撬了起来，还沾沾自喜，但是发现结果是撬起来一块之后，体力值瞬间归零，上一篇我也提到了，其实干这类活也是很有技巧性的，但是上次的是我没学会，可能需要花时间学的，但是这次是LD 用她的纤细胳膊教会我的，我在撬的时候，屏住一口气，双手用力，起，大概是吃好几口奶的力气都用出来了，但是 LD 在我休息的时候，慢慢悠悠的，先把撬棍挤到木条或者混凝土跟下层的缝里，然后往下垫一小块混凝土碎石，然后轻轻松松的扳两下，就撬开了，亏我高中的时候引以为傲的物理成绩，作为物理课代表，这么浅显易懂的杠杆原理都完全不会用到生活里，后面在用这个技巧撬的过程中，真的觉得自己蠢到家了，当然在明白了用点杠杆原理之后，撬地板的活就变得慢慢悠悠，悠哉悠哉的了(其实还是很热的，披着毛巾擦眼睛)。<br>上午的活差不多完了，后面就是把撬出来的混凝土和地板条丢下去，地上铺着不用了的被子，然后就是午饭和午休环节了，午饭换了一家快餐，味道非常可以，下午的活就比较单调了，帮忙清理了上去扔下来的混凝土碎块跟木条，然后稍微打扫了下，老丈人就让我们回家了，接着上次说的，还是觉得比跑步啥的消耗大太多了，那汗流的，一口就能喝完一瓶 500 毫升左右的矿泉水。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>运动</category>
|
|
<category>跑步</category>
|
|
<category>干活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>运动</tag>
|
|
<tag>减肥</tag>
|
|
<tag>跑步</tag>
|
|
<tag>干活</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>解决一个比较奇妙的问题 - leancloud 阅读计数不显示</title>
|
|
<url>/2025/02/09/%E8%A7%A3%E5%86%B3%E4%B8%80%E4%B8%AA%E6%AF%94%E8%BE%83%E5%A5%87%E5%A6%99%E7%9A%84%E9%97%AE%E9%A2%98/</url>
|
|
<content><![CDATA[<p>最近发现博客上那个阅读次数的功能有点问题，因为是基于leancloud的功能，然后前几天打开了下 leancloud.app 这个网站，发现打不开了，以为是不提供服务了，因为当时刚好没精力就没仔细去研究，后面点开具体一篇文章，提示说Counter未初始化，本身这个功能是基于 leancloud 实现的计数器，需要在博客构建的时候初始化本次新增文章的一个类似于键值对的计数对。<br>第一步思考的错误是网站搞错了，leancloud 的官网地址是 leancloud.cn，leancloud 的正确官网是可以打开的。<br>那么第二步的思考是可能换了接口或者调用方式，因为打开以后要求验证手机号码，以为是也加强了审核，需要通过手机绑定后才能继续访问，<br>第二步这个手机验证完成后，发现还是不能够正常使用，开始注意具体的提示信息，是否真的就只是counter没初始化<br>因为我的博客是会同步推送到自己的另一个 gitea 服务，最近应该是入口 traefik 的状态有点异常，导致无法同步到gitea服务，但是不影响正常推送到github的page服务<br>第三步的问题随着这个思考就想到了，因为hexo在部署推送的时候是逐个串行地往目标部署点推送的，比如我有github跟gitea，还有个其他的部署点，那么假如前面gitea的出现了问题，后续的就没法往下推送了，正巧 leancloud 的这个计数服务是通过部署点实现的，在部署日志中会有 </p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">INFO Deploying: leancloud_counter_security_sync</span><br></pre></td></tr></table></figure>
|
|
<p>那么问题就差不多解决了，只是 traefik 这个问题还要看下，尝试把 traefik 的 docker-compose 启动起来，发现 80 端口被占了，<br>这时候就又是一个经典问题了，首先是找到哪个进程占用了这个端口</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">lsof -i:80</span><br></pre></td></tr></table></figure>
|
|
<p>通过这个命令也可以，当然也可以通过 netstat，只是我一直记不住，用的不太多<br>发现是这个 apache2，就是重启ubuntu后默认启动的 apache 的 http 服务器，<br>只是这里需要注意下，它的服务名是 apache2 ，而不是httpd<br>所以可以通过</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">systemctl stop httpd</span><br></pre></td></tr></table></figure>
|
|
<p>来关闭，同时可以用 </p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">systemctl disable httpd</span><br></pre></td></tr></table></figure>
|
|
<p>直接把这个自启动的给干掉，省得每次重启都有这个问题，这一连串的排查还是比较有意思的</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>linux</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>linux</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>解决wsl2中的ubuntu无法通过ssh连接的问题</title>
|
|
<url>/2025/06/22/%E8%A7%A3%E5%86%B3wsl2%E4%B8%AD%E7%9A%84ubuntu%E6%97%A0%E6%B3%95%E9%80%9A%E8%BF%87ssh%E8%BF%9E%E6%8E%A5%E7%9A%84%E9%97%AE%E9%A2%98/</url>
|
|
<content><![CDATA[<p>之前介绍过wsl2的ssh连接问题，但是最近又碰到无法连接的情况，通过一顿查资料<br>发现了个比较特别的方法<br>首先可以通过powershell命令查看目前的端口开放情况</p>
|
|
<figure class="highlight powershell"><table><tr><td class="code"><pre><span class="line"><span class="built_in">Get-NetFirewallRule</span> | <span class="built_in">Where-Object</span> {<span class="variable">$_</span>.LocalPort <span class="operator">-eq</span> <span class="number">22</span> <span class="operator">-or</span> <span class="variable">$_</span>.DisplayName <span class="operator">-like</span> <span class="string">"*SSH*"</span>} | <span class="built_in">Select-Object</span> DisplayName, Enabled, Action, Direction</span><br></pre></td></tr></table></figure>
|
|
<p> 这个是查看的22端口或者名称带有SSH的<br> 还可以直接看22端口的入站规则<br> <figure class="highlight powershell"><table><tr><td class="code"><pre><span class="line"><span class="built_in">Get-NetFirewallRule</span> <span class="literal">-Direction</span> Inbound | <span class="built_in">Where-Object</span> {<span class="variable">$_</span>.LocalPort <span class="operator">-eq</span> <span class="number">22</span>} | <span class="built_in">Select-Object</span> DisplayName, Enabled, Action</span><br></pre></td></tr></table></figure><br>确认下是否已有对应规则，没有的话可以通过下面的命令来添加<br>方法1：允许所有网络的ssh访问</p>
|
|
<figure class="highlight powershell"><table><tr><td class="code"><pre><span class="line"><span class="built_in">New-NetFirewallRule</span> <span class="literal">-DisplayName</span> <span class="string">"WSL2 SSH Inbound"</span> <span class="literal">-Direction</span> Inbound <span class="literal">-Protocol</span> TCP <span class="literal">-LocalPort</span> <span class="number">22</span> <span class="literal">-Action</span> Allow</span><br></pre></td></tr></table></figure>
|
|
<p>当然这样不太安全<br>所以我们最好是指定下来源ip<br>方法二：只允许局域网ip来访问</p>
|
|
<figure class="highlight powershell"><table><tr><td class="code"><pre><span class="line"><span class="built_in">New-NetFirewallRule</span> <span class="literal">-DisplayName</span> <span class="string">"WSL2 SSH LAN"</span> <span class="literal">-Direction</span> Inbound <span class="literal">-Protocol</span> TCP <span class="literal">-LocalPort</span> <span class="number">22</span> <span class="literal">-Action</span> Allow <span class="literal">-RemoteAddress</span> <span class="number">192.168</span>.<span class="number">0.0</span>/<span class="number">16</span>,<span class="number">10.0</span>.<span class="number">0.0</span>/<span class="number">8</span>,<span class="number">172.16</span>.<span class="number">0.0</span>/<span class="number">12</span></span><br></pre></td></tr></table></figure>
|
|
<p>后面的网段可以根据所需限定<br>然后可以在所需机器进行访问验证</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">ssh -v user@192.168.xxx.xxx</span><br></pre></td></tr></table></figure>
|
|
<p>可以访问了的话说明我们这个就已经放行了，通过这个我们也能发现其实wsl2的这种方式一方面是让linux更加完整，但是对于所处的windows宿主机，想进行一些连通访问就会变得更加麻烦，也算是种取舍</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>wsl</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>wsl</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>远程桌面工具rustdesk的私有化部署-mac锁定问题</title>
|
|
<url>/2024/05/19/%E8%BF%9C%E7%A8%8B%E6%A1%8C%E9%9D%A2%E5%B7%A5%E5%85%B7rustdesk%E7%9A%84%E7%A7%81%E6%9C%89%E5%8C%96%E9%83%A8%E7%BD%B2-mac%E9%94%81%E5%AE%9A%E9%97%AE%E9%A2%98/</url>
|
|
<content><![CDATA[<p>上次分享了rustdesk如何私有化部署，但是在实际使用中存在一个小问题，就是如果被连接端是mac的话，在mac进入超时自动锁定或者被主动锁定之后，连接就会进入一个“已连接，等待画面传输”的状态，但实际后续并不会传输画面，而是继续卡在这个无法实际连接的状态<br><img data-src="https://img.nicksxs.me/blog/KO74SB.png"><br>这个问题目前最新版的rustdesk还没有很好的解决，不过可以通过一个小技巧来解决，只是会有一个限制<br>就是需要能够使用vnc连接这个被连接端的mac或者ssh连接<br>第一种是如果能用vnc连接的话，就先连接，然后稍微动下鼠标，然后rustdesk就能获取画面进行下一步操作<br>第二种是通过ssh连接到mac以后，通过一个mac下的小命令 <code>caffeinate</code> 这个命令是mac自带的，可以防止mac<br>进入睡眠状态，具体命令如下</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">caffeinate -u -t 2</span><br></pre></td></tr></table></figure>
|
|
<p><code>-u</code>: 表示用户处于活动状态，此参数将打开屏幕并禁止屏幕进入空闲休眠。</p>
|
|
<p><code>-d</code>：禁止屏幕休眠。</p>
|
|
<p><code>-i</code>：禁止系统空闲休眠。</p>
|
|
<p><code>-m</code>：禁止硬盘进入休眠。</p>
|
|
<p><code>-s</code>：禁止系统进入睡眠状态，此参数仅在插上电源的时候才有效。</p>
|
|
<p><code>-t</code>：指定命令有效的超时值，以秒为单位。<br>这样我们就能从rustdesk获得画面进行操作了，希望后面rustdesk能解决掉这个问题，因为这两种方式也不是很完善，如果只有纯远程的话就没办法了</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>工具</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>工具</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>解决下idea中maven依赖的cannot download sources问题</title>
|
|
<url>/2025/06/15/%E8%A7%A3%E5%86%B3%E4%B8%8Bidea%E4%B8%AD%E4%BE%9D%E8%B5%96%E7%9A%84cannot-download-sources%E9%97%AE%E9%A2%98/</url>
|
|
<content><![CDATA[<p>发现一个比较奇怪的问题，在看一个依赖的源码的时候没法下载sources，idea点击的时候报”cannot download sources”<br>可能有点强迫症，看着缩略过的代码比较不舒服，就查了下这个问题的解决方法</p>
|
|
<h2 id="第一种maven命令"><a href="#第一种maven命令" class="headerlink" title="第一种maven命令"></a>第一种maven命令</h2><p>首先第一种是直接使用maven命令<br>在项目目录下执行</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">mvn dependency:resolve -Dclassifier=sources</span><br></pre></td></tr></table></figure>
|
|
<p>命令，这样就会去下载这些依赖包的sources源码，等待命令构建完成就可以了<br>如果idea自动重建索引了就直接生效了，如果没有则需要choose sources</p>
|
|
<h2 id="第二种idea设置"><a href="#第二种idea设置" class="headerlink" title="第二种idea设置"></a>第二种idea设置</h2><p>还有一种，找到idea的设置中的build工具，maven，import设置，勾选自动下载sources<br><img data-src="https://img.nicksxs.me/idea-auto-download-sources.png"><br>可以勾选sources，就会在下载依赖包的时候就自动下载sources<br>点击idea的maven刷新按钮就会在下载依赖包的时候同代下载sources源码<br>这些前提是本身打包的时候是带上sources源码的，没有用一些混淆加密手段过</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Java</category>
|
|
<category>Maven</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>Java</tag>
|
|
<tag>Maven</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>重看了下《蛮荒记》说说感受</title>
|
|
<url>/2021/10/10/%E9%87%8D%E7%9C%8B%E4%BA%86%E4%B8%8B%E3%80%8A%E8%9B%AE%E8%8D%92%E8%AE%B0%E3%80%8B%E8%AF%B4%E8%AF%B4%E6%84%9F%E5%8F%97/</url>
|
|
<content><![CDATA[<p>周末把《蛮荒记》看完了，前面是发现微信读书有《搜神记》和《蛮荒记》，但是《搜神记》看了会发现很多都是跳段了，不知道为啥，貌似也没什么少儿不宜的情节，所以就上网找了原版来看，为什么看这个呢，主要还是高中的时候看过，觉得写得很不错，属于那时候的玄幻小说里的独一档，基于山海经创造了一个半架空的大荒宇宙，五族帝尊，人物名都是听说过的，而且又能契合部分历史，整个故事布局非常宏大，并且情节矛盾埋得很深，这里就不对具体情节作介绍了，只是聊聊对书中的一些人物和情节的看法感受。</p>
|
|
<p>乌丝兰玛是个贯穿两部，甚至在蛮荒的最后还要再搞事情，极其坚定的自以为是的大 boss，其实除了最后被我们的主人公打败，前面几乎就是无所不能，下了一盘无比巨大的棋，主人公都只是其中一个棋子和意外，但是正如很多反派，一直以来都是背着一个信念，并且这个所谓的信念是比较正义的，只是为了这个正义的信念和目标却做了各种丧尽天良的事情，说起来跟灭霸有点像，为了环保哈哈，相对来说感觉姬远玄也只是个最大牌的工具人，或者说是中间人，深爱的妹妹冰夷也意外被蚩尤怒拿一血。</p>
|
|
<p>但是中间那个赤霞仙子一定要给烈烟石的心上锁，导致最后认不出来蚩尤，也间接导致了蚩尤被杀，如果不考虑最后情节或者推动故事的需求，这个还是我很讨厌的，有点类似于《驴得水》里那个校长，看着貌似是个正常的，做的事情也是正派，但是其实是害人不浅，即使南阳仙子因此被抛进了火山，那也是有贱人在那挑食，并且赤松子是赤飚怒的儿子，烈烟石跟蚩尤又没这层关系，就很像倚天屠龙记里的灭绝师太和极品家丁里的那个玉德仙坊的院主，后者还好一些，前者几乎就是导致周芷若一生悲剧的始作俑者，自己偏执的善恶观，还要给徒弟灌输如此恶毒的理念和让她立下像紧箍咒似的誓言，在人一生中本来就有很多不能如愿的，又被最亲最尊敬的人下了这样的紧箍咒，人生的不幸也加倍了。</p>
|
|
<p>似乎习惯了总要有个总结的，想说的应该是我觉得这些剧也好，书也好，我觉得最坏的人可能是大部分人眼中的一些次要人物，或者至少大 boss 才是最坏的人，当然这个坏也不是严格的二分法，只是我觉得最让我觉得负面的人物，这些人可能看起来情景出现的不多，只是说了很少的话，做了很少的事，但是在我看来却做了最大的恶。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>看书</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>闲聊下乘公交的用户体验</title>
|
|
<url>/2021/02/28/%E9%97%B2%E8%81%8A%E4%B8%8B%E4%B9%98%E5%85%AC%E4%BA%A4%E7%9A%84%E7%94%A8%E6%88%B7%E4%BD%93%E9%AA%8C/</url>
|
|
<content><![CDATA[<p>新年开工开车来杭州，因为没有车位加限行今天来就没开车来了，从东站做公交回住的地方，这班神奇的车我之前也吐槽过了，有神奇的乘客和神奇的司机，因为基本上这班车是从我毕业就开始乘了，所以也算是比较熟悉了，以前总体感觉不太好的是乘坐时间太长了，不过这个也不能怪车，是我自己住得远（离东站），后来住到了现在的地方，也算是直达，并且 LD 比较喜欢直达的，不爱更快却要换乘的地铁，所以坐的频率比较高，也说过前面那些比较气人的乘客，自己不好好戴口罩，反而联合一起上车的乘客诽谤司机，说他要吃人了要打人了，也正是这个司机比较有意思，上车就让戴好口罩，还给大家讲，哪里哪里又有疫情了，我觉得其实这个司机还是不错的，特殊时期，对于这种公共交通，这样的确是比较负责任的做法，只是说话方式，语气这个因人而异，他也不是来伺候人的，而且这么一大车人，说了一遍不行，再说一遍，三遍以上了，嗓门大一点也属于正常的人的行为。<br>还是说回今天要说的，今天这位司机我看着跟前面说的那位有点像，因为上车的时候比较暗没看清脸，主要原因是这位司机开车比较猛，比较急，然后车上因为这个时间点，比较多大学开学来的学生，拎着个行李箱，一开始是前面已经都站满了人，后面还有很多空位，因为后面没地方放行李箱，就因为这样前面站着的有几个就在说司机开慢点，结果司机貌似也没听进去，还是我行我素，过了会又有人说司机开稳一点，就在这个人说完没一会，停在红绿灯路口的车里，就有人问有没有垃圾桶，接着又让司机开门，说晕车太严重了，要下车，司机开了门，我望出去两个妹子下了车，好像在路边草丛吐了，前面开门下车的时候就有人说她们第一次来杭州，可能有点责怪司机开的不稳，也影响了杭州交通给新来杭州的人的感受，说完了事情经过，其实我有蛮多感触，对于杭州公交司机，我大概是大一来了没多久，陪室友去文三路买电脑就晕车，下车的时候在公交车站吐了，可能是从大学开始缺乏锻炼，又饮食不规律，更加容易晕车，大部分晕车我觉得都是我自己的原因，有时候是上车前吃太多了，或者早上起太早，没睡好，没吃东西，反正自己也是挺多原因的，说到司机的原因的话，我觉得可能这班车还算好的，最让我难受的还是上下班高峰的时候，因为经过的那条路是比较重要的主干道，路比较老比较窄，并且还有很多人行道，所以经常一脚油门连带着一脚刹车，真的很难受，这种算是我觉得真的是公交体验比较差的一点，但是这一点呢也不能完全怪公交司机，杭州的路政规划是很垃圾，没看错，是垃圾，所以总体结论是公交还行，主要是路政规划就是垃圾，包括这条主干道这么多人行道，并且两边都是老小区，老年人在上班高峰可能要买菜送娃或者其他事情，在通畅的情况下可能只需要六分钟的路程，有时候因为各种原因，半小时都开不完，扯开去一点，杭州的路，核心的高速说封就封，本来是高架可以直接通到城西，结果没造，到了路本已经很拥挤的时候开始来造隧道，各种破坏，隧道接高架的地方，无尽的加塞，对于我这样的小白司机来说真的是太恶心了，所以我一直想说的就是杭州这个地方房价领先基础设施十年，地铁，高架，高速通通不行，地面道路就更不行了。<br>总结下，其实杭州的真正的公交体验差，应该还是路造成的，对于前面的那两位妹子来说，有可能是她们来自于公交司机都是开的特别稳，并且路况也很好的地方，也或者是我被虐习惯了🤦♂️</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
<category>公交</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>开车</tag>
|
|
<tag>加塞</tag>
|
|
<tag>糟心事</tag>
|
|
<tag>规则</tag>
|
|
<tag>公交</tag>
|
|
<tag>路政规划</tag>
|
|
<tag>基础设施</tag>
|
|
<tag>杭州</tag>
|
|
<tag>高速</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>闲话篇-也算碰到了为老不尊和坏人变老了的典型案例</title>
|
|
<url>/2022/05/22/%E9%97%B2%E8%AF%9D%E7%AF%87-%E4%B9%9F%E7%AE%97%E7%A2%B0%E5%88%B0%E4%BA%86%E4%B8%BA%E8%80%81%E4%B8%8D%E5%B0%8A%E5%92%8C%E5%9D%8F%E4%BA%BA%E5%8F%98%E8%80%81%E4%BA%86%E7%9A%84%E5%85%B8%E5%9E%8B%E6%A1%88%E4%BE%8B/</url>
|
|
<content><![CDATA[<p>在目前的房子也差不多租了四五年了，楼下邻居换了两拨了，我们这栋楼装修了不知道多少次，因为是学区的原因，房子交易的频率还是比较高的，不过比较神奇的我们对门的没换过，而且一直也没什么交集（除了后面说的水管爆裂），就进出的时候偶尔看到应该是住着一对老夫妻，感觉年纪也有个七八十了。</p>
|
|
<p>对对面这户人家的印象，就是对面的老头子经常是我出门上班去了他回来，看着他颤颤巍巍地走楼梯，我看到了都靠边走，而且有几次还听见好像是他儿子在说他，”年假这么大了，还是少出去吧”，说实话除了这次的事情，之前就有一次水管阀门爆裂了，算是有点交集，那次大概是去年冬天，天气已经很冷了，我们周日下午回来看到楼梯有点湿，但是没什么特别的异常就没怎么注意，到晚上洗完澡，楼下的邻居就来敲门，说我们门外的水表那一直在流水，出门一看真的是懵了，外面水表那在哗哗哗地流水，导致楼梯那就跟水帘洞一样，仔细看看是对面家的水表阀门那在漏水，我只能先用塑料袋包一下，然后大冬天（刚洗完澡）穿着凉拖跑下去找物业保安，走到一楼的时候发现水一直流到一楼了，楼梯上都是水流下来，五楼那是最惨的，感觉门框周边都浸透了，五楼的也是态度比较差的让我一定要把水弄好了，但是前面也说了谁是从对门那户的水表阀那出来的，理论上应该让对面的处理，结果我敲门敲了半天对面都没反应，想着我放着不管也不太好，就去找了物业保安，保安上来看了只能先把总阀关了，我也打电话给维修自来水管的，自来水公司的人过了会也是真的来修了，我那会是挺怕不来修，自来水公司的师傅到了以后拿开一看是对面那户的有个阀门估计是自己换上去的，跟我们这的完全不一样，看上去就比较劣质，师傅也挺气的，大晚上被叫过来，我又尝试着去敲门也还是没人应，也没办法，对面老人家我敲太响到时候出来说我吓到他们啥的，第二天去说也没现场了。</p>
|
|
<p>前面的这件事是个重要铺垫，前几天 LD 下班后把厨余垃圾套好袋子放在门口，打算等我下班了因为要去做核酸（hz 48 小时核酸）顺便带下去，结果到了七点多，说对面的老太太在那疯狂砸门了，LD 被吓到了不敢开门，老太太在外面一边砸门一边骂，“你们年轻人怎么素质这么差”（他们家也经常在门口放垃圾，我们刚来住的时候在楼梯转角他们就放这个废弃的洗衣机，每次走楼梯带点东西都要小心翼翼地走，不然都过不去，然后我赶紧赶回去，结果她听到我回家了，还特意开门继续骂，“你们年轻人怎么素质这么差，垃圾放在这里”，我说我们刚才放在这，打算待会做核酸的时候去扔掉，结果他们家老头，都已经没了牙齿，在那瞪大眼睛说，“你们早上就放在这了的，”我说是LD 刚才下班了放的，争论了一会，我说这个事情我们门口放了垃圾，这会我就去扔掉了，但是你们家老太太这么砸门总不太好，像之前门口水管爆掉了，我敲了门没人应，我也没要砸门一定把你们叫醒，结果老头老太说我们的水管从来没换过，不可能破的（其实到这，再往后说就没意思了，跟这么不要脸的人说多了也只是瞎扯），一会又回到这个垃圾的问题，那个老头说“你们昨天就放在这里了的”，睁着眼说瞎话可真是 666，感觉不是老太太拦着点他马上就要冲上来揍我了一样，事后我想想，这种情况我大概只能躺地上装死了，当这个事情发生之前我真的快把前面说的事情（水管阀坏了）给忘了，虽然这是理论上不该我来处理，除非是老头老太太请求我帮忙，这事后面我也从没说起过，本来完全没交集，对他们的是怎么样的人也没概念，总觉得年纪大了可能还比较心宽和蔼点，结果没想到就是一典型的坏人变老了，我说你们这么砸门，我老婆都被吓得不敢开门，结果对面老头老太太的儿子也出来了说，“我们就是敲下门，我母亲是机关单位退休的，所以肯定不会敲门很大声的，你老婆觉得吓到了是你们人生观价值观有问题”，听到这话我差点笑出来，连着两个可笑至极的脑残逻辑，无语他妈给无语开门，无语到家了。对门家我们之前有个印象就是因为我们都是顶楼，这边老小区以前都是把前后阳台包进来的，然后社区就来咨询大家的意见是不是统一把包进来的违建拆掉，还特地上来六楼跟他们说，结果对面的老头就说，“我要去住建局投诉你们”，本来这个事情是违法的，但是社区的意思也是征求各位业主的意见，结果感觉是社区上门强拆了一样，为老不尊，坏人变老了的典范了。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>闲话篇-路遇神逻辑骑车带娃爹</title>
|
|
<url>/2022/05/08/%E9%97%B2%E8%AF%9D%E7%AF%87-%E8%B7%AF%E9%81%87%E7%A5%9E%E9%80%BB%E8%BE%91%E9%AA%91%E8%BD%A6%E5%B8%A6%E5%A8%83%E7%88%B9/</url>
|
|
<content><![CDATA[<p>周末吃完中饭去买菜，没想到碰到这个神（zhi）奇（zhang）大哥带着两个娃，在非机动车道虽然没有像上班高峰车那么多，但是有送外卖，各种叮咚买菜和普通像我这样骑电驴，骑自行车的人，我的情况可能还特殊点，前面说过电驴买了以后本来网上找到过怎么解除限速的，后面看了下，限速 25 虽然慢，但还是对安全很有好处的，我上下班也不赶这个时间，所以就没解除，其他路上的电瓶车包括这位带娃的大哥可能有不少都不符合国标的限速要求或者解除了限速，这些算是铺垫。</p>
|
|
<p>那位大哥，骑电瓶车一前一后带着两个娃，在非机动车道靠右边行驶，肉眼估计是在我右前方大概十几米的距离，不知道是小孩不舒服了还是啥，想下来还是就在跟他爹玩耍，我算是比较谨慎骑车的，看到这种情况已经准备好捏刹车了，但是也没想到这个娃这么神，差不多能并排四五辆电瓶车的非机动车道，直接从他爸的车下来跑到了非机动车道的最左边，前面我铺垫了电瓶车 25 码，换算一下大概 1 秒能前进 7 米，我是直接把刹车捏死了，才勉强避免撞上这个小孩，并且当时的情况本来我左后方有另一个大哥是想从我左边超过去，因为我刹车了他也赶紧刹车。</p>
|
|
<p>现在我们做个假设，假如我刹车不够及时，撞上了这个小孩，会是啥后果呢，小孩人没事还好，即使没事也免不了大吵一架，说我骑车不看前面，然后去医院做检查，负责医药费，如果是有点啥伤了，这事估计是没完了，我是心里一阵后怕。</p>
|
|
<p>说实话是张口快骂人了，“怎么带小孩的”，结果那大哥竟然还是那套话术，“你们骑车不会慢点的啊，说一下就好了啊，用得着这么说吗”，我是真的被这位的逻辑给打败了，还好是想超我车那大哥刹住车了，他要是刹不住呢，把我撞了我怪谁？这不是追尾事件，是 zhizhang 大哥的小孩鬼探头，下个电瓶车就下车，下来就往另一边跑，我们尽力刹车没撞到这小孩，说他没管好小孩这大哥还觉得自己委屈了？结果我倒是想骂脏话了，结果我左后方的的大哥就跟他说“你这么教小孩教得真好，你真厉害”，果然在中国还是不能好好说话，阴阳怪气才是王道，我前面也说了真的是后怕，为什么我从头到尾都没有说这个小孩不对，我是觉得这个年纪的小孩（估摸着也就五六岁或者再大个一两岁）这种安全意识应该是要父母和学校老师一起教育培养的，在路上不能这么随便乱跑，即使别人撞了他，别人有责任，那小孩的生理伤痛和心理伤害，父母也肯定要心疼的吧，另外对我们来说前面也说了，真的撞到了我们也是很难受的，这个社会里真的是自私自利的人太多了，平时让外卖小哥送爬下楼梯送上来外卖都觉得挺抱歉的，每次的接过来都说谢谢，人家也不容易，换在有些人身上大概会觉得自己花了钱就是大爷，给我送上来是必须的。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>阿里云 rds 主从延迟排查</title>
|
|
<url>/2023/12/24/%E9%98%BF%E9%87%8C%E4%BA%91-rds-%E4%B8%BB%E4%BB%8E%E5%BB%B6%E8%BF%9F%E6%8E%92%E6%9F%A5/</url>
|
|
<content><![CDATA[<p>昨天同学问我是不是数据库主从延迟有点高，可能有一分多钟，然后我就去看了rds 的监控，发现主实例上的监控显示的延迟才 1.2 秒，而且是最高 1.2 秒，感觉这样的话应该就没啥问题，然后同学跟我说他加了日志，大致的逻辑是主库数据落库以后就会发一条 mq 消息出来，然后消费者接收到以后回去从库查一下这个数据，结果发现延迟了 90 多秒才查到数据，这种情况比较可能的猜测是阿里云这个监控的逻辑可能是从库在获得第一条同步数据的时候，而不是最终同步完成，但是跟阿里云咨询了并不是，使用的就是 <code>show slave status</code> 结果里的 <code>Seconds_Behind_Master</code> 指标，那这第一种情况就否定掉了，这里其实是犯了个错误，应该去从库看这个延迟的，不过对于阿里云来说在 rds 监控是看不到从库的监控的，只能到 das，也就是阿里云的数据库自治服务可以看到，这里能看到从库的延迟监控，发现的确有这么高，这样就要考虑为啥会出现这种情况，阿里云同学反馈的是这段时间的 iops 很高，并且 cpu 也比较高，让我排查下 binlog，这里就碰到一个小问题，阿里云 rds 的线上实例我们没法在本地连接，并且密码也是在代码里加密过的，去服务器上连接一方面需要装 mysql 客户端，另一方面是怕拉取日志会有性能影响，幸好阿里云这点做的比较好，在 rds 的”备份恢复”–> “日志备份”菜单里可以找到binlog 文件， 在这里其实大致就发现了问题，因为出问题的时间段内升成 binlog 的量大大超过其他时间段，然后通过内网下载后就可以对 binlog 进行分析了，这里我们用到了 mysqlbinlog 工具，主要就是找具体是哪些写入导致这个问题，mysqlbinlog 可以在 mysql 官网下载 mysql 的压缩包，注意是压缩包，这样解压了直接用就好了，不用完整安装 mysql，一般我们需要对 binlog 进行 base64 的反编码，</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">./mysqlbinlog -vv --base64-output=decode-rows mysql-bin.xxxx | less</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>这样查看里面的具体信息，发现是大量的 insert 数据，再经过排查发现同一个 binlog 文件近 500M 全是同一个表的数据插入，再根据表对应的业务查找发现是有个业务逻辑会在这个时间点全量删除后在生成插入数据，后续需要进行优化</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>Mysql</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>mysql</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>难得的大扫除</title>
|
|
<url>/2022/04/10/%E9%9A%BE%E5%BE%97%E7%9A%84%E5%A4%A7%E6%89%AB%E9%99%A4/</url>
|
|
<content><![CDATA[<p>因为房东要来续签合同，记得之前她说要来看看，后来一直都没来成，一方面我们没打扫过也不想被看到，小房子东西从搬进来以后越来越多，虽然不是脏乱差，但也觉得有点不满意干净状态，这里不得不感叹房东家的有钱程度，买了房子自己都没进房子看过，买来只是为了个学籍，去年前房东把房子卖给新房东后，我们还是比较担心会要换房子了，这里其实是个我们在乎的优点略大于缺点的小房子，面积比较小，但是交通便利以及上下班通勤，周边配套也还不错，有个比较大的菜市场，虽然不常去，因为不太会挑不会还价，还是主要去附近一公里左右的超市，可以安静地挑菜，但是说实在的菜场的菜还是比超市新鲜一些。<br>大扫除说实在的住在这边以后就没有一次真正意义上的大扫除，因为平时也有在正常打扫，只有偶尔的厨房煤气灶和厕所专门清理下，平时扫地拖地都有做，但是因为说实在的这房子也比较老了，地板什么的都有明显的老化，表面上的油漆都已经被磨损掉了，一些污渍很难拖干净，而且包括厨房和厕所的瓷砖都是纹路特别多，加上磨损，基本是污渍很多，特别是厨房的，又有油渍，我们搬进来的时候厨房的地就已经不太干净了，还有一点就是虽然不是在乡下的房子，但是旁边有两条主干道，一般只要开着窗没几天就灰尘积起来了，公司的电脑在家两天不到就一层灰，而且有些灰在地上时间久一点就会变成那种棉絮状的，看起来就会觉得更脏，并且地板我们平时就是扫一下，然后拖一下没明显的脏东西跟大灰尘就好了，有一些脏的就很难拖干净。<br>这次的算是整体的大扫除，把柜子，桌子，茶几台，窗边的灰尘都要擦掉，有一些角落还是有蛮多灰尘，当然特别难受的就是电脑那些接口，线缆上的，都杂糅在一块，如果要全都解开了理顺了还是比较麻烦，并且得断电，所以还是尽力清理，但没有全弄开了（我承认我是在偷懒，这里得说下清理了键盘，键盘之前都是放着用，也没盖住，按键缝里就很容易积灰也很难清理，这次索性直接把键全拔了，但是里面的清理也还是挺麻烦，因为不是平板一块，而且还有小孔，有些缝隙也比较难擦进去，只能慢慢地用牙线棒裹着抹布还有棉签擦一下，然后把键帽用洗手液什么的都擦一下洗洗干净，最后晾干了装好感觉就是一把新键盘了，后面主要是拖地了，这次最神奇的就是这个拖地，本来我就跟 LD 吹牛说拖地我是专业的，从小拖到大，有些地板缝边上的污渍，我又是用力来回拖，再用脚踩着拖，还是能把一些原来以为拖不掉的污渍给拖干净了，但是后来的厨房就比较难，用洗洁精来回拖感觉一点都起不来，可能是污渍积了太久了，一开始都想要放弃了，就打算拖干就好了，后来突然看到旁边有个洗衣服的板刷，结果竟然能刷起来，这样就停不下来了，说累是真的非常累，感觉刷一块瓷砖就要休息一会，但是整体刷完之后就是焕然一新的赶脚，简直太有成就感了。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>生活</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>生活</tag>
|
|
<tag>大扫除</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>远程桌面工具rustdesk的私有化部署</title>
|
|
<url>/2024/05/12/%E8%BF%9C%E7%A8%8B%E6%A1%8C%E9%9D%A2%E5%B7%A5%E5%85%B7rustdesk%E7%9A%84%E7%A7%81%E6%9C%89%E5%8C%96%E9%83%A8%E7%BD%B2/</url>
|
|
<content><![CDATA[<p>最近因为利用rustdesk远程操作诈骗太多，所以rustdesk官方直接关闭了国内的公有服务，相对其他比如teamviewer来说，rustdesk还是个比较不错的选择，性能考量也过得去，相对来说在双方都是mac的情况下，不付费的体验来说，比自带的vnc好了很多，vnc目测是直接全量传实时画面数据，带宽占用大，体验差。正巧最近有需求就使用了rustdesk所以记录下部署过程<br>我使用的较早先版本的部署方式，<br>首先需要一台有外网ip的服务器，并且需要使用到以下几个端口，如果使用云服务器，需要在控制台开启这些端口<br>核心端口:</p>
|
|
<figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">TCP 21114-21119</span><br><span class="line">UDP 21116</span><br></pre></td></tr></table></figure>
|
|
<p>测试有没有开启可以使用这个工具网站 <code>CanYouSeeMe.org</code> ，或者用另一台机器<code>telnet</code>下<br>需要部署的服务有两个<br>hbbs - RustDesk ID/Rendezvous server id服务<br>hbbr - RustDesk Relay server 中继服务<br>这里使用docker来部署，比较简单，先拉取镜像，没错就一个</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sudo docker image pull rustdesk/rustdesk-server</span><br></pre></td></tr></table></figure>
|
|
<p>这边建议创建个rustdesk目录，进入目录后再启动下面的服务<br>先启动hbbs</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sudo docker run --name hbbs -p 21115:21115 -p 21116:21116 -p 21116:21116/udp -p 21118:21118 -v `pwd`:/root -td --net=host rustdesk/rustdesk-server hbbs</span><br></pre></td></tr></table></figure>
|
|
<p>使用 <code>--net=host</code> 是为了让server拿到连接进入的真实来源ip<br>然后再启动hbbr</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sudo docker run --name hbbr -p 21117:21117 -p 21119:21119 -v `pwd`:/root -td --net=host rustdesk/rustdesk-server hbbr</span><br></pre></td></tr></table></figure>
|
|
<p>接下来进到刚才建的目录，会看到有个一对公私钥，把公钥内容保存下来<br>然后就是客户端登录了，以mac为例<br><img data-src="https://img.nicksxs.me/blog/0IRnc8.png"><br>点击这边三点，再点击修改中继设置<br><img data-src="https://img.nicksxs.me/blog/sCw4lP.png"><br>在弹出窗口<br><img data-src="https://img.nicksxs.me/blog/wJ6DYV.png"><br>ID服务器跟中继服务器框填入前面部署服务的服务器外网ip，在Key框里把刚才保存的公钥填进去<br>每个需要连接的客户端都需要这么设置，然后就可以使用自己的服务器作为ID跟中继服务器了，不会受官方关停国内服务影响</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>工具</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>工具</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>通过显卡来给gpt-oss做个加速</title>
|
|
<url>/2025/09/21/%E9%80%9A%E8%BF%87%E6%98%BE%E5%8D%A1%E6%9D%A5%E7%BB%99gpt-oss%E5%81%9A%E4%B8%AA%E5%8A%A0%E9%80%9F/</url>
|
|
<content><![CDATA[<p>之前在我的MacBookPro里使用gpt-oss，因为总的内存只有18g，加上系统本身和各种后台程序的占用，剩下的一般就一半左右，然后这个模型大小就有13g左右，运行起来是很吃力的<br>正好我这有个服役比较久的windows笔记本，带有3060显卡，只是显存是6g的，想试下看能不能通过显卡来给它加加速，这里我使用了lm studio来运行gpt-oss 20b模型<br>这个lm studio可以配置有多少层transformer运行在gpu上，<br><img data-src="https://img.nicksxs.me/gpt_offload.png"><br>我们可以大致除一下<br>这个gpt-oss 20b是有24层，官标的显存需求是16g，我们有6g显存，大约可以加载8~9层的样子，并且lm studio也自动算出来是加载8层<br>通过这个配置我们运行的gpt-oss能够得到勉强可用的程度<br><img data-src="https://img.nicksxs.me/gpt_oss_token.png"><br>能达到4.x个token的生成速度，还是比mac上要把所有程序全关了，清掉很多token来得方便<br>实测将gpu卸载层数调整到9是勉强还可以运行，再往上就加载不了了，显存小没办法<br>当然我们还可以继续用 unsloth 进行微调<br>例如参考 <a href="https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/gpt-oss-(20B)-Fine-tuning.ipynb">https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/gpt-oss-(20B)-Fine-tuning.ipynb</a><br>同时可以通过nvidia-smi来查看显存占用<br><img data-src="https://img.nicksxs.me/gpu_memory.png"><br>过程中倒是没看到显存完全占满，可以通过nvitop再进行观察<br>能够在gpu中加载部分层的确能把大模型的生成速度提升一些，但是还是受限于显存大小，最好是能有显存大于16g的显卡可以尝试测一下</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>LLM</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>LLM</tag>
|
|
</tags>
|
|
</entry>
|
|
<entry>
|
|
<title>逐步迁移-用rclone将腾讯云cos迁移到cloudflare的r2</title>
|
|
<url>/2025/03/30/%E9%80%90%E6%AD%A5%E8%BF%81%E7%A7%BB-%E7%94%A8rclone%E5%B0%86%E8%85%BE%E8%AE%AF%E4%BA%91cos%E8%BF%81%E7%A7%BB%E5%88%B0cloudflare%E7%9A%84r2/</url>
|
|
<content><![CDATA[<p>因为上次那个问题，所以打算把图库迁移到靠谱一些的cloudflare上，这里用到了rclone这个很强大的工具</p>
|
|
<p>在开始迁移前，先做一下准备</p>
|
|
<ol>
|
|
<li>分别在腾讯云和 Cloudflare 平台申请 Access Key 和 Secret Key</li>
|
|
<li>安装 rclone 工具（可从 <a href="https://rclone.org/downloads/">rclone 官网</a> 下载）</li>
|
|
</ol>
|
|
<h2 id="配置远程存储"><a href="#配置远程存储" class="headerlink" title="配置远程存储"></a>配置远程存储</h2><h3 id="配置步骤"><a href="#配置步骤" class="headerlink" title="配置步骤"></a>配置步骤</h3><ol>
|
|
<li>打开终端，输入 <code>rclone config</code> 开始远程存储配置</li>
|
|
<li>按 <code>n</code> 创建新的远程存储</li>
|
|
</ol>
|
|
<h3 id="配置腾讯云-COS"><a href="#配置腾讯云-COS" class="headerlink" title="配置腾讯云 COS"></a>配置腾讯云 COS</h3><ol>
|
|
<li>输入远程存储名称，例如 <code>cos</code></li>
|
|
<li>选择存储类型，选择 <code>Amazon S3</code> 兼容模式（选项编号可能因 rclone 版本而异）</li>
|
|
<li>选择服务提供商为腾讯云 COS</li>
|
|
<li>输入您的 <code>access_key_id</code> 和 <code>secret_access_key</code>（从腾讯云控制台获取）</li>
|
|
<li>选择相应的端点 API（根据您的存储桶所在地区选择北京或上海等）</li>
|
|
<li>配置访问权限：可以选择仅资源所有者拥有完全访问权限，或根据您的安全需求选择其他选项</li>
|
|
<li>选择存储类型（通常选择标准存储）</li>
|
|
<li>确认配置</li>
|
|
</ol>
|
|
<h3 id="配置-Cloudflare-R2"><a href="#配置-Cloudflare-R2" class="headerlink" title="配置 Cloudflare R2"></a>配置 Cloudflare R2</h3><ol>
|
|
<li>再次输入 <code>n</code> 创建新的远程存储</li>
|
|
<li>输入远程存储名称，例如 <code>r2</code></li>
|
|
<li>同样选择 <code>Amazon S3</code> 兼容模式</li>
|
|
<li>选择服务提供商为 Cloudflare R2</li>
|
|
<li>输入您的 <code>access_key_id</code> 和 <code>secret_access_key</code>（从 Cloudflare 控制台获取）</li>
|
|
<li>输入 R2 的端点 URL（通常格式为 <code>https://<account_id>.r2.cloudflarestorage.com</code>）</li>
|
|
<li>配置访问权限</li>
|
|
<li>选择存储类型</li>
|
|
<li>确认配置</li>
|
|
</ol>
|
|
<h2 id="执行迁移操作"><a href="#执行迁移操作" class="headerlink" title="执行迁移操作"></a>执行迁移操作</h2><p>完成配置后，使用以下命令执行迁移：</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">rclone copy cos:mystore-xxxxxxx/img/ r2:img/ --progress</span><br></pre></td></tr></table></figure>
|
|
|
|
<p>上述命令将把腾讯云 COS 中 <code>mystore-xxxxxxx</code> 存储桶下的 <code>img</code> 目录中的所有文件复制到 Cloudflare R2 的 <code>img</code> 存储桶中。添加的 <code>--progress</code> 参数可以显示迁移进度。</p>
|
|
<h2 id="高级选项"><a href="#高级选项" class="headerlink" title="高级选项"></a>高级选项</h2><p>为了获得更好的迁移体验，您可以考虑以下高级选项：</p>
|
|
<ul>
|
|
<li>使用 <code>--transfers=N</code> 参数设置并行传输数量，加快迁移速度</li>
|
|
<li>使用 <code>--checkers=N</code> 参数设置并行检查数量</li>
|
|
<li>使用 <code>--dry-run</code> 参数测试迁移过程而不实际复制文件</li>
|
|
<li>使用 <code>--log-file=FILE</code> 参数将迁移日志保存到文件中</li>
|
|
</ul>
|
|
<p>例如：</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">rclone copy cos:mystore-xxxxxxx/img/ r2:img/ --progress --transfers=4 --checkers=8 --stats=10s</span><br></pre></td></tr></table></figure>
|
|
|
|
<h2 id="验证迁移结果"><a href="#验证迁移结果" class="headerlink" title="验证迁移结果"></a>验证迁移结果</h2><p>迁移完成后，可以使用以下命令验证两边的文件是否一致：</p>
|
|
<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">rclone check cos:mystore-xxxxxxx/img/ r2:img/ --one-way</span><br></pre></td></tr></table></figure>
|
|
|
|
<h2 id="结语"><a href="#结语" class="headerlink" title="结语"></a>结语</h2><p>通过 rclone 这一强大工具，可以轻松实现云存储服务之间的数据迁移。Cloudflare R2 提供了稳定可靠的对象存储服务，同时其无出站流量费用的特性也使其成为图库存储的理想选择。</p>
|
|
]]></content>
|
|
<categories>
|
|
<category>体验</category>
|
|
</categories>
|
|
<tags>
|
|
<tag>恶意盗刷</tag>
|
|
</tags>
|
|
</entry>
|
|
</search>
|